Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smoke Tests Codegen Working #3155

Merged
merged 51 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
feeaccd
v1 codegen
sbera87 Oct 15, 2024
76ef3f9
updates
sbera87 Oct 16, 2024
f877902
updates
sbera87 Oct 16, 2024
ae0b5b2
codegen fixes
sbera87 Oct 16, 2024
1bba920
add service filter in configuration
sbera87 Oct 16, 2024
4dec4b3
add deps
sbera87 Oct 16, 2024
fbfb5c6
deps
sbera87 Oct 16, 2024
18f0386
projections test
sbera87 Oct 17, 2024
4b5a7f0
working codegen from script
sbera87 Oct 17, 2024
0233376
codegen script updates
sbera87 Oct 18, 2024
346c1f0
working codegen with script
sbera87 Oct 18, 2024
dc6da8d
ignore temp files for smithy codegen
sbera87 Oct 18, 2024
fb8414b
cleanup + codegen fixes
sbera87 Oct 18, 2024
872c19b
tweaks
sbera87 Oct 18, 2024
049a36d
remove temp files
sbera87 Oct 18, 2024
320e03d
remove temp files
sbera87 Oct 18, 2024
5ca8987
This release launches the CreateIntegration, DeleteIntegration, Descr…
aws-sdk-cpp-automation Oct 15, 2024
6b97b8c
Add support for the new optional bucket-region and prefix query param…
aws-sdk-cpp-automation Oct 16, 2024
785f364
Update API model
aws-sdk-cpp-automation Oct 17, 2024
1273eb9
This is an Amazon ECS documentation only update to address tickets.
aws-sdk-cpp-automation Oct 17, 2024
97e4f61
Don't continue polling input stream in case it is closed or in a bad …
SergeyRyabinin Oct 17, 2024
cbad9dc
Revert "Don't continue polling input stream in case it is closed or i…
SergeyRyabinin Oct 18, 2024
b49ca3a
Update API model
aws-sdk-cpp-automation Oct 18, 2024
ac1b079
Added converse support for custom imported models
aws-sdk-cpp-automation Oct 18, 2024
9cb4e54
codegen script updates
sbera87 Oct 18, 2024
d44f0eb
This release launches the CreateIntegration, DeleteIntegration, Descr…
aws-sdk-cpp-automation Oct 15, 2024
7a3f507
Add support for the new optional bucket-region and prefix query param…
aws-sdk-cpp-automation Oct 16, 2024
f16a2c1
merge
sbera87 Oct 20, 2024
b8bc77f
codegen fixes with cas
sbera87 Oct 21, 2024
1cf31c7
remove stray lines
sbera87 Oct 21, 2024
d3100ca
changes to get all clients to compile
sbera87 Oct 21, 2024
f9852cb
handle legacy and new service code generation
sbera87 Oct 22, 2024
5ae8434
fixes
sbera87 Oct 22, 2024
03cf379
cleanup
sbera87 Oct 22, 2024
b27da0e
cleanup
sbera87 Oct 23, 2024
31c8317
utilize importer for dependencies
sbera87 Oct 23, 2024
0c96460
handle num
sbera87 Oct 23, 2024
3f9f29b
cleanup + resolve PR comments
sbera87 Oct 24, 2024
c75a2d7
restore
sbera87 Oct 24, 2024
efa5bb4
remove helper for case conversion
sbera87 Oct 24, 2024
1be4d61
cleanup unused functions
sbera87 Oct 24, 2024
b27e018
remove commented lines
sbera87 Oct 25, 2024
e1c9608
PR comments fix
sbera87 Oct 25, 2024
febfcaf
cleanup
sbera87 Oct 25, 2024
c1a3350
cleanup
sbera87 Oct 25, 2024
aec431a
cleanup
sbera87 Oct 25, 2024
d336778
cleanup
sbera87 Oct 25, 2024
d2c23a6
formatting fixes
sbera87 Oct 25, 2024
b599f31
remove exceptions
sbera87 Oct 25, 2024
90f1772
PR feedback changes
sbera87 Oct 28, 2024
69600c5
remove catch exceptions
sbera87 Oct 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
updates
  • Loading branch information
sbera87 committed Oct 16, 2024
commit f877902207b3544cc30587721c8e8117a0773c0d
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,11 @@ public String generate()
set_compiler_flags($${PROJECT_NAME})
set_compiler_warnings($${PROJECT_NAME})

target_include_directories($${PROJECT_NAME} PUBLIC
$${CMAKE_CURRENT_SOURCE_DIR}/../../src/aws-cpp-sdk-$L/include)

target_link_libraries($${PROJECT_NAME} $${PROJECT_LIBS})
""", awsTestSrc, awsTestSrc);
""", awsTestSrc, awsTestSrc,folderNamespace);

write("""
if (AUTORUN_UNIT_TESTS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack. i left this code in case we ever want to auto run smoke tests. I can remove it for now

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,23 +51,51 @@ public class SmokeTestsParser implements Runnable{
final private SmithyCodegenAdapter codegenAdapter;
final private PluginContext context;
final private SymbolProvider symbolProvider;
final private Map<ShapeId, String> operationToServiceMap;
final private Map<String, ServiceShape> serviceShapeMap;

public SmokeTestsParser(PluginContext context)
{
this.context = context;
this.model = context.getModel();
codegenAdapter = new SmithyCodegenAdapter(model);
this.codegenAdapter = new SmithyCodegenAdapter(model);
this.symbolProvider = new CppSymbolVisitor(model);


operationToServiceMap = new HashMap<>();
serviceShapeMap = new HashMap<>();

// Iterate over all Service shapes in the model and create map of operation to
model.getServiceShapes().stream().forEach(serviceShape -> {
String serviceName = serviceShape.getId().getName();
serviceShape.getAllOperations().stream().forEach(operation -> {
this.operationToServiceMap.put( operation, serviceName);
this.serviceShapeMap.put(serviceName,serviceShape );
});
});
}

private String getServiceName(ShapeId serviceShapeId)
public static String removeSpaces(String input)
{
String clientName = serviceShapeId.getName();
System.out.println("clientName="+clientName);
return input.replace(" ", "");
}

public static String toKebabCase(String input) {
// Remove leading and trailing spaces, replace spaces with hyphens, and convert to lowercase
return input.trim().toLowerCase().replace(" ", "-");
}

private String getServiceName(ServiceShape serviceShape) throws Exception
{
if(!serviceShape.getTrait(ServiceTrait.class).isPresent())
{
//System.err.println(String.format("No service trait detected in service shape with name=%s",serviceShape.getId().getName()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented out code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

throw new Exception(String.format("No service trait detected in service shape with name=%s",serviceShape.getId().getName()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont throw exception, everyon up chain has to check it, never throw or check Exception, subclass or throw a RuntimeException

Copy link
Contributor Author

@sbera87 sbera87 Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is your recommendation how I should handle this? I understand that stack unwinding will happen on exception but what do i return when service trait is missing. I can throw a Runtimeexception

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they have a concept of "checked and unchecked exceptions" in java.
additionally, simple regular Exception is too broad and the exception handling error code must catch it and do it's logic accordingly to the exception string message (vs simply type of the exception).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack. I will switch to runtime exception as it is one of those cases where I have to throw if service trait not found

}

//this.model.getShape(serviceShapeId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented out code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still commented out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack. Oversight

String clientName = serviceShape.getTrait(ServiceTrait.class).get().getSdkId();

int underscoreIndex = clientName.indexOf('_');
// Check if underscore exists, otherwise return the whole string
clientName = underscoreIndex != -1 ? clientName.substring(0, underscoreIndex) : clientName;
return clientName;
}

Expand Down Expand Up @@ -213,22 +241,9 @@ else if(serviceShape.getTrait(HttpBearerAuthTrait.class).isPresent())

//model contains information from all the smithy files
//extract services to smoke tests
public Map<ShapeId, List<SmokeTestData> > extractServiceSmokeTests()
public Map<ServiceShape, List<SmokeTestData> > extractServiceSmokeTests()
{
Map<ShapeId, List<SmokeTestData> > serviceSmokeTestsMap = new HashMap<>();

Map<ShapeId, String> operationToServiceMap = new HashMap<>();
Map<String, ServiceShape> serviceShapeMap = new HashMap<>();

// Iterate over all Service shapes in the model and create map of operation to
model.getServiceShapes().stream().forEach(serviceShape -> {
String serviceName = serviceShape.getId().getName();
serviceShape.getAllOperations().stream().forEach(operation -> {
operationToServiceMap.put( operation, serviceName);
serviceShapeMap.put(serviceName,serviceShape );
});
});

Map<ServiceShape, List<SmokeTestData> > serviceSmokeTestsMap = new HashMap<>();

//first filter operations that have smoke test trait
//on those operation shapes, find service trait
Expand All @@ -241,9 +256,7 @@ public Map<ShapeId, List<SmokeTestData> > extractServiceSmokeTests()
SmokeTestsTrait smokeTestsTrait = operationShape.getTrait(SmokeTestsTrait.class).get();
//get serviceShape
String serviceName = operationToServiceMap.get(operationShape.getId());

ServiceShape serviceShape = serviceShapeMap.get(serviceName);

System.out.println("OperationShape: " + operationShape.getId().getName());
System.out.println("serviceName: " + serviceName);

Expand All @@ -252,13 +265,13 @@ public Map<ShapeId, List<SmokeTestData> > extractServiceSmokeTests()
operationShape,
serviceShape );
//add to tests for the same service
if(serviceSmokeTestsMap.containsKey(serviceShape.getId()))
if(serviceSmokeTestsMap.containsKey(serviceShape))
{
serviceSmokeTestsMap.get(serviceShape.getId()).addAll(tests);
serviceSmokeTestsMap.get(serviceShape).addAll(tests);
}
else
{
serviceSmokeTestsMap.put(serviceShape.getId(), tests);
serviceSmokeTestsMap.put(serviceShape, tests);
}

});
Expand All @@ -274,23 +287,31 @@ public void run(){
SmokeTestsSourceDelegator delegator = new SmokeTestsSourceDelegator(this.context.getFileManifest(), this.symbolProvider);
SmokeTestsCMakeDelegator cmakedelegator = new SmokeTestsCMakeDelegator(this.context.getFileManifest(), this.symbolProvider);

Map<ShapeId, List<SmokeTestData> > smoketests = extractServiceSmokeTests();
Map<ServiceShape, List<SmokeTestData> > smoketests = extractServiceSmokeTests();

//make service specific folder
smoketests.entrySet().stream().forEach(entry -> {
String serviceName = getServiceName(entry.getKey());
Path relativePath = Paths.get( serviceName );
System.out.println(String.format("path=%s",relativePath.toString() + "/"+ serviceName + "SmokeTests.cpp"));

delegator.useFileWriter( relativePath.toString() + "/"+ getServiceName(entry.getKey()) + "SmokeTests.cpp", serviceName, writer -> {
System.out.println("generating smoke test source code");
writer.generate(entry.getValue());
});

cmakedelegator.useFileWriter( relativePath.toString() + "/"+ "CMakeLists.txt", serviceName, writer -> {
System.out.println("generating smoke test cmake code");
writer.generate();
});
ServiceShape serviceShape = entry.getKey();

try{
String client = getServiceName(serviceShape);

Path relativePath = Paths.get( toKebabCase(client) );
System.out.println(String.format("path=%s",relativePath.toString() + "/"+ removeSpaces(client) + "SmokeTests.cpp"));

delegator.useFileWriter( relativePath.toString() + "/"+ removeSpaces(client) + "SmokeTests.cpp", client, writer -> {
System.out.println("generating smoke test source code");
writer.generate(entry.getValue());
});

cmakedelegator.useFileWriter( relativePath.toString() + "/"+ "CMakeLists.txt", client, writer -> {
System.out.println("generating smoke test cmake code");
writer.generate();
});
}
catch (Exception e) {
System.err.println("Exception detected=" + e.toString());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove catch or re-throw runtime exception so the generation will actually fail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

}
});

delegator.flushWriters();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ protected void defineTestFixture()
write("public:").
write("static const char ALLOCATION_TAG[];").
dedent().
write("}");
write("const char $LSmokeTestSuite::ALLOCATION_TAG[] = \"$LSmokeTest\"",clientNamespace,clientNamespace);
write("};");
write("const char $LSmokeTestSuite::ALLOCATION_TAG[] = \"$LSmokeTest\";",clientNamespace,clientNamespace);
}

protected void defineTestCase(SmokeTestData test)
Expand Down
Loading