JavaParser
per generare, modificare e analizzare codice Java
Federico Tomassetti
The JavaParser family
JavaParser JavaSymbolSolver
a.k.a. JP a.k.a. JSS
Is this stuff mature?
JavaParser is a project with a long history, contributions from over 50
persons, and basically it works.
JavaSymbolSolver is much younger and it works decently enough.
Until it does not.
Yes, it supports all of Java
Even the crazy things you all forgot about…
Yes, it supports all of Java
Even the crazy things no one actually used…
What JavaParser does?
JavaParser… parse Java code into a Java AST
package [Link];
CompilationUnit
class A {
int field;
} PackageDecl ClassDecl
FieldDecl
PrimitiveType
What JavaParser does?
JavaParser unparse an AST into code
CompilationUnit
package [Link];
PackageDecl ClassDecl
class A {
int field;
FieldDecl }
PrimitiveType
Hello, JavaParser!
// Get a compilation unit
[Link](myFile)
[Link](code)
// Or an expression
[Link](”1 + 2”)
// Or a statement
[Link](”if (a) b = 1;”)
Isn’t JP enough?
int foo;
public void aMethod(int foo) {
foo = 1;
}
public void anotherMethod() {
foo = 1;
}
To JP these two statements looks the same: they produce the same
AST nodes.
It is the assignment of a thing named ”foo”, no idea what that thing is
Isn’t JP enough?
public void print1(String foo) {
[Link](foo);
}
public void print2(int foo) {
[Link](foo);
}
To JP these two statements looks the same: they produce the same
AST nodes.
It is the call of a method named “print”, no idea which signature that
has
Isn’t JP enough?
class A { }
public void creator1() {
new A();
}
public void creator2() {
class A { }
new A();
}
To JP these two statements looks the same: they produce the same
AST nodes.
It is the instantiation of a class named “A”, no idea where it is defined
What JavaSymbolSolver does?
JavaSymbolSolver resolves symbols in the JavaParser AST
package [Link];
CompilationUnit
class C {
D field;
} PackageDecl ClassDecl
FieldDecl
ReferenceType
package [Link];
class D {
}
Relationship JP & JSS
Certain methods in the AST requires additional intelligence.
[Link]();
JSS not enabled JSS enabled
[Link]();
JSS not enabled JSS enabled
Hello, JavaSymbolSolver!
// 1) Prepare JavaParser
TypeSolver typeSolver = /* configure where to look */;
ParserConfiguration parserConfiguration =
new ParserConfiguration().setSymbolResolver(
new JavaSymbolSolver(typeSolver));
JavaParser parser = new JavaParser(parserConfiguration);
// 2) Parse using the advanced API
CompilationUnit compilationUnit =
[Link](ParseStart.COMPILATION_UNIT,
new StreamProvider(new FileInputStream(myFile)))
.getResult().get();
// 3) Use the AST… with some extra functionalities
Hello, JavaSymbolSolver!
CompilationUnit cu = /* we have an AST */
// JSS can calculate the type of any expression
[Link]();
myExpression resolved type
1+2 int
2.0 * 3 double
"foo".charAt(0) char
“foo”.length() int
new A() [Link].A
Hello, JavaSymbolSolver!
CompilationUnit cu = /* we have an AST */
// JSS can figure out which method has been called
ResolvedMethodDeclaration methodDeclaration =
[Link]();
[Link]();
methodCall Method declaration
[Link](0) [Link](int)
[Link](“a”) [Link](String)
"foo".charAt(0) [Link](int)
“foo”.length() [Link]()
new LinkedList<String>().size() [Link]()
new LinkedList<String>().toString() [Link]()
Hello, JavaSymbolSolver!
CompilationUnit cu = /* we have an AST */
// JSS knows if two types are assignables
type1 = [Link]();
type2 = [Link]().getType();
if ([Link](type2)) { … }
type1 type2 result
int double false
double int true
Collection<Int> List<Int> true
Collection<Double> List<Int> false
Collection<? extends String> List<String> true
Comments attribution
void foo() { // comment1
// comment1 int a =
int a = 1 + 2;
1 + 2; // comment2
}
CompilationUnit cu = [Link](code);
ExpressionStmt expressionStmt =
[Link]([Link]).get() ;
[Link]("Comment on the expression statement: "
+ [Link]().get().getContent());
Comments attribution
void foo() {
// comment1 1 + 2 // comment2
int a =
1 + 2; // comment2
}
VariableDeclarationExpr expr =
(VariableDeclarationExpr)expressionStmt
.getExpression();
VariableDeclarator variableDeclarator =
[Link]().get(0);
Expression init = variableDeclarator
.getInitializer().get();
[Link]("Comment on the initializer: "
+ [Link]().get().getContent());
Can you show me the AST?
Node node = parseBodyDeclaration(
"public Class<? extends String> methodName(String arg) {}");
// If your grandpa needs the AST
[Link](new XmlPrinter(true).output(node));
// Because JavaScript has won
[Link](new JsonPrinter(true).output(node));
// Also hipsters need to see an AST
[Link](new YamlPrinter(true).output(node));
// To generate a diagram with Graphviz
[Link](new DotPrinter(true).output(node));
Can you show me the AST?
root(Type=MethodDeclaration):
body(Type=BlockStmt):
type(Type=ClassOrInterfaceType):
name(Type=SimpleName):
identifier: "Class"
typeArguments:
- typeArgument(Type=WildcardType):
extendedType(Type=ClassOrInterfaceType):
name(Type=SimpleName):
identifier: "String"
name(Type=SimpleName):
identifier: "methodName"
parameters:
- parameter(Type=Parameter):
isVarArgs: "false"
name(Type=SimpleName):
identifier: "arg"
type(Type=ClassOrInterfaceType):
name(Type=SimpleName):
identifier: "String"
Lexical preservation
JavaParser can do pretty printing
String code = "class MyClass{int a;float b;void bar(){}}";
CompilationUnit cu = [Link](code);
[Link]([Link]());
class MyClass {
int a;
float b;
void bar() {
}
}
Lexical preservation
JavaParser can do also do lexical preservation
String code = "class MyClass {int a;float b;void bar(){}}";
ParserConfiguration parserConfiguration =
new ParserConfiguration()
.setLexicalPreservationEnabled(true);
CompilationUnit cu = new JavaParser(parserConfiguration)
.parse(ParseStart.COMPILATION_UNIT,
new StringProvider(code)
).getResult().get();
[Link]([Link]());
class MyClass {int a; void bar(){}}";
Configuring JavaSymbolSolver
JSS needs one thing: that you tell it where to look for classes.
• CombinedTypeSolver to group different type solvers
• AarTypeSolver look into an aar package
• JarTypeSolver look into a jar package
• JavaParserTypeSolver look into a directory of Java files
• MemoryTypeSolver for testing purposes
• ReflectionTypeSolver use reflection (useful to java(x).* classes)
Configuring JavaSymbolSolver
A typical usage:
CombinedTypeSolver typeSolver = new CombinedTypeSolver(
new ReflectionTypeSolver(),
new JavaParserTypeSolver(new File("src/main/java")),
new JavaParserTypeSolver(new File("src/test/java")),
new JarTypeSolver("libs/[Link]"),
new JarTypeSolver("libs/[Link]"));
JavaParser to run queries
Setup: let’s consider the code from Hamcrest
// The directory where there is the code
File hamcrestCoreDir = new File(
"src/main/resources/JavaHamcrest-src/hamcrest-
core/src/main/java");
// Configure the Symbol Solver
CombinedTypeSolver typeSolver = new CombinedTypeSolver(
new ReflectionTypeSolver(),
new JavaParserTypeSolver(hamcrestCoreDir));
// Use our Symbol Solver while parsing
ParserConfiguration parserConfiguration =
new ParserConfiguration()
.setSymbolResolver(new JavaSymbolSolver(typeSolver));
JavaParser to run queries
Setup: let’s consider the code from Hamcrest
// Parse all source files
SourceRoot sourceRoot = new
SourceRoot([Link]());
[Link](parserConfiguration);
List<ParseResult<CompilationUnit>> parseResults =
[Link]("");
// Now get all compilation units
List<CompilationUnit> allCus = [Link]()
.filter(ParseResult::isSuccessful)
.map(r -> [Link]().get())
.collect([Link]());
JavaParser to run queries
Question: How many methods take more than 3 parameters?
long n = getNodes(allCus, [Link])
.stream()
.filter(m -> [Link]().size() > 3)
.count();
[Link]("N of methods with 3+ params: " + n);
Answer: 11
JavaParser to run queries
Question: What are the three top classes with most methods?
getNodes(allCus, [Link])
.stream()
.filter(c -> ![Link]())
.sorted([Link](o ->
-1 * [Link]().size()))
.limit(3)
.forEach(c ->
[Link]([Link]() + ": " +
[Link]().size() + " methods"));
Answer: CoreMatchers: 35 methods
BaseDescription: 13 methods
IsEqual: 9 methods
JavaParser to run queries
Question: What is the class with most ancestors?
ResolvedReferenceTypeDeclaration c = getNodes(allCus,
[Link])
.stream()
.filter(c -> ![Link]())
.map(c -> [Link]()) JSS at work here
.sorted([Link](o ->
-1 * [Link]().size()))
.findFirst().get();
List<String> ancestorNames = [Link]()
.stream()
.map(a -> [Link]())
.collect([Link]());
[Link]([Link]() + ": " +
[Link](", ", ancestorNames));
Answer: [Link]: [Link],
[Link], [Link], [Link],
[Link], [Link]
JavaParser to identify patterns
private static boolean isClassUsingSingleton(
ClassOrInterfaceDeclaration c) {
List<VariableDeclarator> fields = [Link]()
.stream()
.filter(f -> [Link]()
&& [Link]())
.map(FieldDeclaration::getVariables)
.flatMap(Collection::stream)
.filter(v -> isThisClass(c, [Link]().getType()))
.collect([Link]());
…
}
JavaParser to identify patterns
private static boolean isClassUsingSingleton(
ClassOrInterfaceDeclaration c) {
…
List<Pair<MethodDeclaration, VariableDeclarator>> pairs =
[Link]()
.stream()
.filter(m -> [Link]()
&& [Link]()
&& isThisClass(c, [Link]().resolve())
&& [Link]().isPresent())
.filter(m -> fieldReturned(m, fields).isPresent())
.map(m -> new Pair<>(m, fieldReturned(m,
fields).get()))
.collect([Link]());
return ![Link]();
}
JavaParser to identify patterns
private static boolean isThisClass(
ClassOrInterfaceDeclaration classOrInterfaceDeclaration,
ResolvedType type) {
return [Link]()
&& [Link]().getQualifiedName()
.equals(classOrInterfaceDeclaration
.resolve().getQualifiedName());
}
JavaParser to identify patterns
private static Optional<VariableDeclarator> fieldReturned(
MethodDeclaration methodDeclaration,
List<VariableDeclarator> fields) {
if ([Link]().get()
.getStatements().size() != 1) {
return [Link]();
}
Statement statement = [Link]()
.get().getStatement(0);
if (![Link]() ||
![Link]()
.getExpression().isPresent()) {
return [Link]();
}
…
}
JavaParser to identify patterns
private static Optional<VariableDeclarator> fieldReturned(
MethodDeclaration methodDeclaration,
List<VariableDeclarator> fields) {
…
Expression expression = [Link]()
.getExpression().get();
if (![Link]()) {
return [Link]();
}
Optional<VariableDeclarator> field = [Link]()
.filter(f -> [Link]()
.equals([Link]()
.getNameAsString()))
.findFirst();
return field;
}
JavaParser to identify patterns
We are working on the Matcher library to reduce the complexity, it is in the early stages
This gives you a list of pairs name-type for all the properties in your bean.
JavaParser for automated
refactoring
A new version of a library comes up and a deprecated method named oldMethod is
replaced by newMethod. The new method takes 3 parameters: the first one as
oldMethod but inverted and the third one is a boolean, which we want to be always
true
getNodes(allCus, [Link])
.stream()
.filter(m -> [Link]()
.getQualifiedSignature()
.equals("[Link]([Link],
int)"))
.forEach(m -> [Link](replaceCallsToOldMethod(m)));
JavaParser for automated
refactoring
A new version of a library comes up and a deprecated method named oldMethod is
replaced by newMethod. The new method takes 3 parameters: the first one as
oldMethod but inverted and the third one is a boolean, which we want to be always
true
public MethodCallExpr replaceCallsToOldMethod(
MethodCallExpr methodCall) {
MethodCallExpr newMethodCall = new MethodCallExpr(
[Link]().get(), "newMethod");
[Link]([Link](1));
[Link]([Link](0));
[Link](new BooleanLiteralExpr(true));
return newMethodCall;
}
JavaParser to generate code
CompilationUnit cu = new CompilationUnit();
[Link]("[Link]");
ClassOrInterfaceDeclaration book = [Link]("Book");
[Link]("String", "title");
[Link]("Person", "author");
JavaParser to generate code
[Link]([Link])
.addParameter("String", "title")
.addParameter("Person", "author")
.setBody(new BlockStmt()
.addStatement(new ExpressionStmt(new AssignExpr(
new FieldAccessExpr(
new ThisExpr(), "title"),
new NameExpr("title"),
[Link])))
.addStatement(new ExpressionStmt(new AssignExpr(
new FieldAccessExpr(
new ThisExpr(), "author"),
new NameExpr("author"),
[Link]))));
[Link]([Link]());
JavaParser to generate code
[Link]("getTitle", [Link]).setBody(
new BlockStmt().addStatement(
new ReturnStmt(new NameExpr("title"))));
[Link]("getAuthor", [Link]).setBody(
new BlockStmt().addStatement(
new ReturnStmt(new NameExpr("author"))));
[Link]([Link]());
JavaParser to generate code
package [Link];
public class Book {
String title;
Person author;
public Book(String title, Person author) {
[Link] = title;
[Link] = author;
}
public void getTitle() {
return title;
}
public void getAuthor() {
return author;
}
}
JavaParser: what can we use it for?
What we can do Why could we do it
Generate new Java code Stop writing boilerplate
Because large refactoring are
Modifying existing code
boring and error-prone
So we can answers and data on
which to take decision.
Running queries on code
Also, we can enforce our own
rules
JavaParser: Visited
Book on JavaParser and
JavaSymbolSolver, from the core
committers.
Avalaible for 0+ $
Currently 900+ readers
[Link]
Federico Tomassetti
