I posted an answer to someone’s question as below on Stackoverflow.

The question:
I have an application written in Java. In is stored in several files. It uses different classes with different methods. The code is big and complicated. I think it would be easier to understand the code if I have a graphical model of the code (some kind of directed graph). Are there some standard methods for visualization of code. 

My answer:

This is an old question, but as this is coming up in Google searches, I am adding my response here so that it could be useful to the future visitors. Let me also disclose that I am the author of MaintainJ.

Don’t try to understand the whole application

Let me ask you this - why do you want to understand the code? Most probably you are fixing a bug or enhancing a feature of the application. The first thing you should not try to do is to understand the whole application. Trying to understand the entire architecture while starting afresh on a project just overwhelms you.

Believe me when I say this - developers with 10+ years of solid coding experience may not understand how certain parts of the application work even after working on the same project for more than an year (assuming they are not the original developers). They may not understand how the authentication works or how the transaction management works in the application. I am talking about typical enterprise applications with 1000 to 2000 classes and using different frameworks.

Two important skills required to maintain large applications

Then how do they survive and are paid big bucks? Experienced developers usually understand what they are doing; meaning, if they are to fix a bug, they will find the location of the bug, then fix it and make sure that it does not break the rest of the app. If they need to enhance a feature or add a new feature, most of the time, they just have to imitate an existing feature that does a similar thing.

There are two important skills that help them to do this.

1. They are able to analyze the impact of the change(s) they do while fixing a bug. First they locate the problem, change the code and test it to make sure that it works. Then, because they know the Java language well and the frameworks ‘well enough’, they can tell if it will break any other parts of the app. If not, they are done.
2. I said that they simply need to imitate to enhance the application. To imitate effectively, one needs to know Java well and understand the frameworks ‘well enough’. For example, when they are adding a new Struts Action class and adding to the configuration xml, they will first find a similar feature, try to follow the flow of that feature and understand how it works. They may have to tweak a bit of the configuration (like the ‘form’ data being in ‘request’ than in ’session’ scope). But if they know the frameworks ‘well enough’, they can easily do this.

The bottom line is, you don’t need to understand what all the 2000 classes are doing to fix a bug or enhance the app. Just understand what’s needed.

Focus on delivering immediate value

So am I discouraging you from understanding the architecture? No, not at all. All I am asking you is to deliver. Once you start on a project and once you have set up the development environment on your PC, you should not take more than a week to deliver something, however small it may be. If you are an experienced programmer and don’t deliver anything after 2 weeks, how can a manager know if you really working or reading sports news?

So, to make life easier for everyone, deliver something. Don’t go with the attitude that you need to understand the whole application to deliver something valuable. It’s completely false. Adding a small and localized Javascript validation may be very valuable to the business and when you deliver it, the manager feels relieved that he has got some value for his money. Moreover, it gives you the time to read the sports news.

As time passes by and after you deliver 5 small fixes, you would start to slowly understand the architecture. Do not underestimate the time needed to understand each aspect of the app. Give 3-4 days to understand the authentication. May be 2-3 days to understand the transaction management. It really depends on the application and your prior experience on similar applications, but I am just giving the ballpark estimates. Steal the time in between fixing the defects. Do not ask for that time.

When you understand something, write notes or draw the class/sequence/data model diagrams.

Diagrams

Haaa…it took me so long to mention diagrams . I started with the disclosure that I am the author of MaintainJ, the tool that generates runtime sequence diagrams. Let me tell you how it can help you.

The big part of maintenance is to locate the source of a problem or to understand how a feature works.

MaintainJ generated sequence diagrams show the call flow and data flow for a single use case. So, in a simple sequence diagram, you can see which methods are called for a use case. So, if you are fixing a bug, the bug is most probably in one of those methods. Just fix it, ensure that it does not break anything else and get out.

If you need to enhance a feature, understand the call flow of that feature using the sequence diagram and then enhance it. The enhancement may be like adding an extra field or adding a new validation, etc. Usually, adding new code is less risky.

If you need to add a new feature, find some other feature similar to what you need to develop, understand the call flow of that feature using MaintainJ and then imitate it.

Sounds simple? It is actually simple, but there will be cases where you will be doing larger enhancements like building an entirely new feature or something that affects the fundamental design of the application. By the time you are attempting something like that, you should be familiar with the application and understand the architecture of the app reasonably well.

Two caveats to my argument above

1. I mentioned that adding code is less risky than changing existing code. Because you want to avoid changing, you may be tempted to simply copy an existing method and add to it rather than changing the existing code. Resist this temptation. All applications have certain structure or ‘uniformity’. Do not ruin it by bad practices like code duplication. You should know when you are deviating from the ‘uniformity’. Ask a senior developer on the project to review the changes. If you must do something that does not follow the conventions, at least make sure that it’s local to a small class (a private method in a 200 line class would not ruin the application’s esthetics).
2. If you follow the approach outlined above, though you can survive for years in the industry, you run the risk of not understanding the application architectures, which is not good in the long run. This can be avoided by working on bigger changes or by just less Facebook time. Spend time to understand the architecture when you are a little free and document it for other developers.

Conclusion

Focus on immediate value and use the tools that deliver that, but don’t be lazy. Tools and diagrams help, but you can do without them too. You can follow my advice by just talking some time of a senior developer on the project.

–Choudary Kothapalli

I nominated MaintainJ for the Eclipse Community Awards 2012 under the Best Modeling Product category. They ask a few questions on the product. I am posting my responses here with the relevant links.

Product Summary
MaintainJ automatically generates UML sequence and class diagrams for a single use case. The sequence diagrams show the exact runtime call and data flow for a use case. They also show the runtime SQL calls made regardless of the database frameworks used. Users can dynamically explore the sequence diagrams and filter out unwanted details. The class diagrams show the dependencies between the classes involved in the use case.

The diagrams can be exported into UML2 models that can be imported into other UML2 compliant tools like Rational Software Architect. MaintainJ drastically reduces the effort required to analyze, understand, document and enhance complex Java applications.

List 5 of the product’s features:
1. Generates runtime sequence and class diagrams for a single use case.
2. The sequence diagrams show the runtime context of a call (the call arguments, return value and state of the called object).
3. Shows the runtime SQL calls made during the use case regardless of the database frameworks used.
4. The sequence and class diagrams can be exported into UML2 models, which can be imported into other UML2 compliant tools like Rational Software Architect.
5. Supports runtime impact analysis – Users can exactly determine the use cases impacted by a change to a Java class, method or database table or field.

How does this product help users?
1. Using MaintainJ, Java developers can quickly analyze, understand, document and enhance large Java code bases. The MaintainJ demo video is recorded on a Java application that has 8000+ classes, runs on Tomcat and MySQL and uses Spring and Hibernate frameworks.

2. MaintainJ generates detailed runtime sequence and class diagrams for a single use case. The arguments and return value of the call are shown in the sequence diagram. The runtime SQL calls made during a use case, regardless of the database frameworks used, are shown. All these details help developers to quickly troubleshoot a problem or to analyze and enhance the application.

3. MaintainJ supports tracing applications deployed across multiple JVM’s. The call trace captured on different JVM’s can be merged to view the end-to-end call flow across JVM’s. For example, when an application calls a web service running in a different JVM, the call flow across the JVMs is shown in a single sequence diagram. The calls in each JVM are shown in a different color.

4. MaintainJ supports runtime impact analysis - Users can capture the runtime call trace for all the use cases of the application and use that information to exactly determine the use cases impacted by a change to a Java class, method or database table or field.

5. Users can filter out the unwanted details from the MaintainJ generated diagrams and then export the diagrams as UML2 model files. These models can be imported into other UML2 compliant tools like Rational Software Architect (RSA) where they can be edited for enhancing the application.

6. Any UML diagrams generated at runtime can be verbose and difficult to read. MaintainJ provides simple user interface to dynamically explore the sequence diagrams. Users can also search for a class, method or database table or field used in the use case.

7. MaintainJ offers various options to filter out unwanted details from the generated diagrams and to view them at the level of abstraction that the user wants. For example, a use case in a web application typically spans the web tier, business tier and data access tier. Users can apply predefined filters on the diagram to view just the business tier classes. In addition, MaintainJ shows the interactions between the application classes only (no framework or library classes are shown).

8. Users can easily troubleshoot multi-threaded applications using MaintainJ - The sequence diagram shows the runtime interactions between threads in different colors. This helps developers to quickly troubleshoot a multi-threaded application, which would be hard to do using a traditional debugger.

9. MaintainJ integrates seamlessly with JUnit - Users can generate a sequence diagram for every test case and check the sequence diagram for any troubleshooting later.

10. MaintainJ saves lot of effort for teams that maintain large Java applications - Different developers in a team repeatedly spend time to understand the same use cases because there are no tools to quickly document and share that knowledge. By facilitating documentation and sharing of the “Maintenance Experience” within the team using the generated UML diagrams, MaintainJ exponentially reduces the total maintenance effort.

What is unique about this product?
The following features are unique to MaintainJ. No other tool currently available in the market supports these features.1. Generating the runtime sequence diagrams for a use case which show the runtime call context (call arguments, return value and state of the called object)

2. Showing the actual runtime SQL statements made during the use case, regardless of the database frameworks used.

3. Tracing multiple applications running on different JVM’s and generating a sequence diagram that shows the end-to-end call flow across JVM’s.

4. Performing runtime impact analysis, using which the user can exactly determine the use cases impacted by a change to a Java class, method or database table or field.

5. Offering multiple ways to filter out redundant details from the generated diagrams and exporting the diagrams as UML2 models, which in turn can be imported into other UML2 compliant tools like Rational Software Architect.

Date of product’s introduction:
March 1st, 2007

–Choudary Kothapalli.

When tracing the sample application Alfresco, I got an exception like below.

Caused by: com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
javax.xml.bind.JAXBElement does not have a no-arg default constructor.
this problem is related to the following location:
at javax.xml.bind.JAXBElement
at static final javax.xml.bind.JAXBElement org.alfresco.repo.audit.model._3.ObjectFactory.createAudit_aroundBody
at org.alfresco.repo.audit.model._3.ObjectFactory

This kind of exceptions arise when we are trying to trace class files generated at runtime (jaxb, cglib, etc.). The quick fix is to exclude the problem causing class from the aop.xml file as below. This FAQ explains the location of aop.xml and how to edit it.

<exclude within=”org.alfresco.repo.audit.model._3.ObjectFactory”/>

A few other exceptions that could be resolved using the same approach are below:

29-Nov-2011 17:47:45 org.aspectj.weaver.tools.Jdk14Trace error
SEVERE: xxx/xxx/XXXDaoImpl$$EnhancerByCGLIB$$dd2b8471
java.lang.NullPointerException
at org.aspectj.weaver.bcel.LazyMethodGen.remap(LazyMethodGen.java:1415)

Caused by: java.lang.VerifyError: (class: xxx/xxx/XXXFactory$$EnhancerByCGLIB$$43dfa63e, method: init$_aroundBody2 signature: (XXXFactory$$EnhancerByCGLIB$$43dfa63e;Lorg/aspectj/lang/JoinPoint;)V) Unable to pop operand off an empty stack

–Choudary Kothapalli.

I answered a question on Stackoverflow, related to the UML diagrams. I am reproducing it here with some changes.

Only a few high-level UML  diagrams survive the life cycle of the application

We need models to represent the software design. But even in large projects of about 500 man-months, I observed that only 3-4 sequence diagrams really matter and have a chance of surviving the entire life cycle of the application. Those 3-4 sequence diagrams (and class diagrams that represent their static time relationships), usually represent the high level design of the application.

Or, look at it this way:

Any decent enterprise application will not have 20 generic call flows. There will be one or two generic (or abstract) call flows, which all the concrete use cases implement. Let us take a simple Struts / EJB application. The generic flow will be something like - an action class calling a validator and then calling a stateless session bean, which in turn calls a domain class, which will call a DAO. All the use cases of the application just implement this flow with concrete classes that are specific to that use case.

Do you agree?

If you do not, I would like to hear about applications that have 20 different generic call flows and survived for 5 years after the first release.

If you agree with me, we are boiling down to 3-4 class and sequence diagrams even for large enterprise applications comprising several thousand classes.

It is hard to find good UML documentation that is kept in sync with the evolving code

You might say that in addition to the few high-level diagrams, you want to document all the use cases of the system for training or documentation purposes. During my last 14 years of experience in the real enterprise software world, I don’t remember seeing well ‘maintained’ UML documentation. First of all, good documents are difficult to produce and are not found that often. Secondly, they are out of sync with the code most of the time. Most of my experience is with large banks, insurance companies, Auto companies, etc. Those environments are just too hectic and their resources are limited (really? Are we talking banks? Yes, difficult to believe, but true) for ‘maintaining’ good documentation.

So am I suggesting that we get rid of UML?

No. We need visual models to represent complex systems. Human brains seem to be at their best when processing visuals. The visual cortex, which is responsible for processing the visual images, is the largest system in the human brain.

So what is a reasonable solution to easily produce and maintain UML models?

1. Probably we are better off using the current crop of UML tools to draw those 3-4 high-level UML diagrams. If you hate using them, check option 3 below.
2. For the diagrams at the next level of abstraction (any useful models should have different levels of abstraction), generate the UML from source code. You can generate both class and sequence diagrams.
3. In this age of agile methodologies, why not just write the shell classes and generate those 3-4 high-level UML class and sequence diagrams as well? This way there won’t be any UML to maintain at all.

The source code is the truth - generate the UML from the code

Can you argue against the statement that the code is the truth? If you do not, why not generate the models from the source code itself? I am not suggesting the round-trip engineering, by the way. I am just suggesting a one way trip - from code to models.

Problems with the generated UML

There are 2 main problems however with the generated UML.

1. When we hand draw a class diagram, we show the relations between the classes involved in a scenario. Most existing class diagram generating tools allow the user to drop the Java classes (the source code) into the tool and the tool automatically shows the relations between the classes. The problem with this approach is, how does one know about the classes involved in a scenario to begin with?
2. The second problem is the verboseness of the generated diagrams. There are tools available to generate runtime sequence and class diagrams for a scenario. But the diagrams are often very verbose and defeat the purpose of models, which is to highlight the important aspects and filter out unimportant details.

How does MaintainJ address the above problems?

1. MaintainJ generates diagrams for a single use case. So the MaintainJ generated class and sequence diagrams are for a single scenario.
2. MaintainJ offers various options to filter out unimportant details from the diagrams.  A powerful relevant feature in MaintainJ is to ‘exclude’ a class. When we hand draw the diagrams, we could leave out the facade like classes and show a direct call from a Controller class in the web tier to the DAO in the business tier for the sake of clarity, though in reality the call flows through a ‘front controller’ or ‘Session facade’, etc. The ‘Exclude class’ feature of MaintainJ allows one to do exactly the same in the  generated UML diagrams.

–Choudary Kothapalli

So what is

• ‘Runtime Impact Analysis’ or
• ‘Runtime Dependency Analysis’ or
• ‘Dynamic Change Impact Analysis’?

Why is it necessary to begin with? Aren’t we fine with what we have now?

I cannot answer these questions better than Ben Rowland did on his blog titled Runtime Dependency Analysis. Please check that blog to appreciate the power of this new feature in MaintainJ.

My explanation of how MaintainJ addresses this problem is here. I will add screenshots of this feature on the screenshots page of the web site.

This release also adds the ability to see the class fields and methods in a class diagram. Showing all the fields and methods in each class will make the diagram very cumbersome and difficult to read. The chosen solution is to show them in the tool tip when the mouse is hovered over a class.

–Choudary Kothapalli.

Here is a document with detailed instructions to apply MaintainJ on a Tomcat application. The steps in this document should work for other application servers as well.

This document is produced by Chris Velado at Periscope Holdings Inc. Thanks a lot, Chris!

Chris is using MaintainJ on a J2EE application that contains around 4000 classes. The application is based on Struts and uses SQL Server, Oracle 10 and 11g databases.

Choudary Kothapalli.