A Deep Dive into ClassLoader and Reflection — Dynamic Typing and Runtime Modifiable Classes in Java
Two powerful Java features to have in your programming arsenal.
I like to think of programming paradigms as territories. From imperative to declarative programming, each territory has strict laws of conduct regarding what's lawful and what's not. But it's more often than not that developers find themselves in a territory where they have to do something that's frowned upon or illegal. In these cases, they start looking for loopholes and ways to exploit the system to fulfill their needs. They sometimes discover laws that permit actions that oppose the core values of the system. In this article, I aim to show how two esoteric Java features can allow for useful antipatterns in the Java world.
The OOP Territory
Object-oriented programming is one of the most commonly taught and used programming models. It specifies that logic should be built around program-defined data types. When designing programs in Java, programmers first design the core data types needed in the program then incorporate logic into their classes and define how these classes interact with each other.
While this approach certainly has well-acclaimed benefits, a statically typed language such as Java lacks flexibility. It requires strict definitions for data types at compile time. This is not always feasible as developers can often encounter situations where they can’t anticipate at the time of development the exact data types their program will deal with. Moreover, Java programs do not have the ability to evolve their classes at runtime. This can also be a problem in a lot of cases. Consider the following design problems:
Designing Frameworks
A framework, in simple terms, is a code base that facilitates development by providing a structure of components that framework users can build on, a skeleton of sorts. User-defined types can be plugged into the framework for customization. For example, a test engineer might realize that they often implement the same steps for testing a service’s API with variance in code mainly in the datatypes representing the service itself. They recognize that a lot of their code is reusable and could be generalized into a framework that would be used in future service testing. Pure OOP provides a challenge in this case, which is that they have to build logic for ambiguous data types. A helpful framework is one that is general enough that it could be used to test a wide range of services. This fundamentally goes against statically typed languages such as Java which require data types to be manipulated through defined classes.
Designing for Hot Swapping
Hot-swappable Software is software that allows for altering its components at runtime without an execution pause. For example, it's often very costly for a business to take its application down for code updates and hence it might attempt to design apps whose APIs still remain modifiable after deployment. The first challenge faced in this problem is that similar to frameworks, logic must be built around modifiable alterable components and hence ambiguous data types. Moreover, this problem has the added complexity of loading new code at runtime which is a separate additional challenge.
The above two examples show two challenging problems in pure Java object-oriented programming:
- The ability to design for ambiguous data types, those are data types whose classes are unknown and can't be defined at development time.
- The ability to evolve data types by modifying their classes at runtime.
I explored a pairing of two Java features, reflection and class loaders, to solve the above two problems and allow for incredibly flexible Java programs.
This article is quite lengthy and dense, here’s everything discussed as a reference:
Contents:
- Reflection Overview
- Practical Example: JUnit Framework (Reflection)
- ClassLoader System Overview
- Practical Example: Adding Plugin Functionality to a Java Application (Reflection + Class Loaders)
- Practical Example: Modifying Classes at runtime (Reflection + Class Loaders)
- Conclusion
Java Reflection
Perhaps the most marketed feature of Java is the fact that it's a statically typed language.
Static Typing and Invocation
Given some Class Foo:
public class Foo{
public void doSomething(){
System.out.println("I did something");
}Normally, the method doSomething() is invoked as follows:
Foo foo = new Foo();
foo.doSomething();The above code is an example of static typing and invocation. The variable foo is said to be statically typed as its type is resolved to the Foo type at compile time. Similarly, the compiler links the doSomething() method invocation to the instance method defined in the Foo Class. During static typing and invocation, things like variables and method invocations can be viewed as if they commit to certain defined types before execution. This requires class definitions to come first for successful compilation.
Naturally, generic frameworks that are designed to work with any type without restricting to interfaces can’t use static typing and invocation. Instead, what enables such frameworks in Java is an obscure style of programming known as reflective programming. Reflective programming allows breaking the statically typed nature of Java programs. It pushes type assignment to runtime instead of compile time, this is known as dynamic typing.
Reflective Programming
According to the Java Reflection in Action (by Ira R. Forman, Nate Forman):
Reflection is the ability of a running program to examine itself and its software environment, and to change what it does depending on what it finds.
I would classify a developer's main work into two categories, the first is the creative aspect in designing robust maintainable programs. The second is the programming equivalent to mechanical labor work. For instance, developers often write code that depends on external modules or components. When an external module is modified, it's not uncommon for a developer to find themselves going through their code and rewriting their method calls to work with the updated module.
Reflective programming can allow a program to do that instead. Instead of programmers having to do manual tedious tasks like refactoring code, patching JARs, and modifying method invocations, Reflection allows writing programs that can make choices usually made by humans like choosing between class X and class Y or invoking a new method instead of an old one.
For that to make sense, one must understand how classes exist inside the Java Virtual Machine (JVM). The JVM is an isolated environment through which Java programs are executed. For the most part, it's not aware of its host device and its file system, meaning that it's not aware of the .java or .class files that are created during development and compilation (more on this in the class loaders section). The JVM has its own internal memory through which it stores all the necessary data it needs at runtime; this of course includes data about the program's classes. Its memory is composed of multiple components (Method Area, Heap Area, Stack Area, PC registers, Native Method Area)
The Heap Area
The heap area of the JVM is a dynamic memory space that stores the program's current objects, it’s where objects live and exist as entities. It's mainly divided into three areas: young generation, old generation, and Metaspace.
- Young Generation: This part stores newly created objects.
- Old Generation: This part stores long-lived objects, these are objects that survived a certain number of garbage collection rounds in the young generation before being promoted to the old generation.
- MetaSpace: This part of the heap is dedicated to storing special types of objects known as metaobjects, these are objects which represent the program's metadata. It also stores method bytecode.
For Java to behave the way we expect it to, the JVM must track metadata about the program which allows it to execute correctly. For example, for a particular class, the JVM must store info such as its access modifiers, methods, and their types (static method or instance method). For each method, it stores info such as its number of parameters and their types, and the methods return type. The JVM stores a program's metadata the way it knows best, though objects referred to as metaobjects. Java creators defined a special set of classes used internally by the JVM, these classes represent components of the program and give access to them. Things like a program’s classes, interfaces, and methods are modeled as objects that live in MetaSpace (Lives true to its name, everything in Java is an object, even its classes!).
The java.lang package defines a class for type Class:
public final class Class<T> extends Object implements Serializable, GenericDeclaration, Type, AnnotatedElementThis class defines the metaobject that represents a class. Every user-defined class will have a corresponding Class object instantiated for it in the JVM. The class Class defines a lot of interesting methods, most of which give information about the class’s structure:
public Field[] getFields() {..}
/* Returns an array containing Field objects representing all the accessible
public fields of the class or interface represented by this Class object.*/
public Annotation[] getAnnotations() {..}
/* Returns annotations that are present on the class. */
public Method[] getMethods() {..}
/* Returns an array containing Method objects representing all the public
methods of the class or interface represented by this Class object */
public T newInstance(){..}
/* Creates a new instance of the class represented by this Class object. */Similarly, java.lang defines class Method which defines metaobjects representing a method. It also contains a handful of APIs that can provide information about the structure of the particular underlying method modeled by a Method object:
public final class Method extends Executablepublic Annotation [] getDeclaredAnnotations(){..}
/* Returns annotations that are present on this method. */
public Class<?> getDeclaringClass(){..}
/* Returns the Class object representing the class or interface that
declares the method represented by this Method object */
public Class<?>[] getParameterTypes(){..}
/* Returns an array of Class objects that represent the methods parameter
types */
public Class<?> getReturnType(){..}
/* Returns a Class object that represents the return type of the method r
epresented by this Method object. */
public Object invoke(Object obj, Object... args){..}
/* Invokes the underlying method represented by this Method object,
given an instance of the method's class and an array of Objects as
the arguments */.I would encourage you to explore these classes’ APIs and other metaobjects like Field and Annotation through the official documentation. The collective metaobjects in the MetaSpace represent the program itself.
A Java type is alive at runtime. It's not a static entity but rather a living breathing object in the JVM.
While it has a dedicated part of the heap, metaobjects in the MetaSpace are not restricted to internal JVM use only, they are exposed to running programs. A program can perform self-inspection by accessing the metaobjects that represent it. Moreover, metaobjects give access to the part of the program they represent. As shown above, the class Class defines a newInstance()method through which it can return an object instance of the class it represents. Similarly, class Method defines theinvoke() method through which a method it represents can be invoked.
Going back to the reflection definition:
Reflection is the ability of a running program to examine itself and its software environment (by accessing metaobjects such as Class objects representing its classes and discovering their internal structure), and to change what it does depending on what it finds (as they can choose to invoke the part of a program represented by a particular metaobject).
okay... but how does that exactly come into play?
Dynamic Typing and Invocation
As I hinted earlier in the section, the main need for reflection comes when dynamic typing and invocation are needed. Invoking the doSomething() method of the Foo class can be done alternatively using reflection as follows:
String className ="org.example.Foo";
String methodName = "doSomething";
try {
//get the foo object's Class(reflectively Foo's Class object)
Class cls = Class.forName(className);
// Reflectively create a new instance of foo
Object obj = cls.getDeclaredConstructor().newInstance();
//get method doSomething
Method method= cls.getDeclaredMethod(methodName);
//reflectively invoke method doSomething
method.invoke(obj);
}catch(Exception e){
}The above code performs dynamic typing and invocation. Given specified className and methodName variables, it reflectively gets the Class object of the target class and retrieves the Method object of the target method through which it invokes the method. Notice how setting the two variables to a different class name and method name will result in the same code executing a different method as it isn’t coupled to any particular type at compile time.
and finally….
A Practical Example: JUnit
JUnit is a popular automation testing framework for Java applications. It automates many aspects of testing like displaying aggregated test result reports or running repeatable test cases. What’s useful in JUnit is that it's generic, it can invoke test methods of any user-defined class, and users don’t even need to adhere to an interface when designing their test classes. JUnit mainly relies on Annotations for configuration. Annotations are part of the Java language that is used to add metadata or extra information to Java programs without having any direct effect on the code they annotate. JUnit defines the @Test annotation to “mark” test methods, meaning that JUnit knows which user methods to run through this annotation. For example, a user can define the following class:
public class TestsBatch {
public int sum (int a, int b){
return a+b;
}
@Test
public void test1() {
assertEquals(10, sum(3,7));
}
@Test
public void test2() {
assertEquals(100, sum(20,80));
}
}JUnit runs methods test1() and test2() since they are annotated with the @Test annotation. What enables JUnit to work as intended is its extensive use of reflection. It uses reflection to discover user-defined classes at runtime and inspects them to find methods having the @Test annotation. It then dynamically invokes these methods. The base logic that operates the framework is actually relatively simple to implement using the reflection API. Its logic is as follows:
- Discover/loop over all user-defined classes.
- For each class, get its methods and inspect them.
- For each method, check it has the
@Testannotation. If it does then this method needs to be invoked and its results processed.
The above steps are shown in the code below:
File[] files = new File("src/test/java/test").listFiles();
//loop over files in path where user defined classes exist
for (File file : files) {
String fileName = file.getName();
//get Class object corresponding to current file
Class c = Class.forName("test." + fileName.substring(0, fileName.indexOf(".")));
//check that class is neither an interface or enum
if (!c.isInterface() && !c.isEnum()){
//get methods belonging to the class
Method[] methods = c.getDeclaredMethods();
for (Method method : methods) {
//check if @test annotation exist on the method
Annotation annotation= method.getAnnotation(Test.class);
if(annotation!=null){
//instantiate new instance of current class
Object obj = c.getDeclaredConstructor().newInstance();
// invoke current method
Object result = method.invoke(obj);
//do necessary processing with the result object
}
}
}Note: the above code is an example I wrote and does not belong to JUnit. However, JUnit does use reflection in its operations.
While Reflection sounds initially unintuitive (and arguably produces the ugliest code), it proves to be a very useful feature when the statically typed nature of Java is restricting. Moreover, it allows developers to design dynamic systems. It also allows the automation of a lot of laborious aspects of programming. In the next section, I will explore another interesting feature of the JVM, its class loaders, which when paired with reflection allow for fundamentally open expandable systems.
Class Loaders
In Object Oriented Programming, the unit of software is a class, it represents a logical entity or block. How classes are loaded into the JVM isn’t usually of much concern or interest, classes just conveniently exist when needed. However, learning the internals of class loading in Java is useful as The JVM’s internal class loading system is actually exposed to programmers, who can intercept the class loading process and change the behavior of the JVM when needed.
This has a variety of benefits and practical uses. One of them is controlling the exact class files that get loaded into the JVM, this can prevent dependency loading issues and conflicts. Another benefit to class loaders is that they allow introducing new behavior to a running program by dynamically loading new classes, this is the basis for creating plugin architectures and expandable programs in Java. Before diving into the internals of the class loading system in Java, one must understand Java’s execution model.
The JVM
Java takes a hybrid approach between compilation and interpretation. A Java class is first defined by a programmer in a .java file format. The Java compiler then transforms it into bytecode generating the .class file. Byte code is an intermediate language which is understood by the JVM.
Java programs are executed through the Java Virtual Machine (JVM). The idea behind the Java virtual machine is that it acts like an abstract virtual computer that creates an isolated environment for running Java programs. A running JVM is mostly unaware of its host computer and defines a strict protocol for loading program files from the computer’s file system and into it for execution.
The JVM consists of 3 distinct areas: ClassLoader System, Runtime Memory/Data Area, and Execution Engine. The Class Loader subsystem is the only component of the JVM that deals with the host device’s file system. It's responsible for locating class files and loading classes into the JVM. When requested to load a class, the class loader system reads its class file bytecode into the JVM then constructs the equivalent Class object and stores its methods bytecode in the heap (Recall that classes exist inside the JVM as Class objects). The steps are shown in the following diagram:
The process of loading classes in the JVM is dynamic, meaning that classes aren’t preloaded but rather loaded on demand, this is known as lazy loading. On program start, the JVM loads the class containing the main() method and all the classes it references. Afterward, whenever a new type is encountered, the ClassLoader system is asked to load it. Users can define their own class loaders and integrate them with the class loading system for custom class loading.
The Class Loader Subsystem
The Class Loader subsystem is composed of multiple class loader components working together, each capable of loading certain classes. It shouldn’t come as a surprise that class loaders are modeled as classes in the JVM. An individual class loader is an object of a special internal Java class (Java being Java!). The java.lang package defines abstract type ClassLoader which is essentially a class that is responsible for loading other classes into the JVM.
public abstract class ClassLoader extends ObjectConcrete subclasses of ClassLoader define different types of class loaders. The main difference between ClassLoader concrete types is in the types of classes they load and the paths through which they look for class files. According to the Java docs:
Given the binary name of a class, a class loader should attempt to locate or generate data that constitutes a definition for the class. A typical strategy is to transform the name into a file name and then read a “class file” of that name from a file system.
Class loaders are structured as a hierarchy with parent-child relationships, meaning that a class loader object is a child to another class loader and is possibly a parent to some other class loader(s). There are three main class loaders used at runtime:
- Bootstrap Class Loader, also known as the Primordial class loader and null class loader, kickstarts the runtime environment. It's the root class loader and is the only class loader that isn’t a Java class but rather platform-dependent machine code that loads the core classes for the basic Java Runtime Environment (JRE), this includes classes in the
java.utiland thejava.langpackages from the jdk/jre/lib/ path. Thejava.langpackage contains core classes like Object, String, and ClassLoader class which enables the initialization of all subsequent class loaders. - Extension Class Loader is a child of the Bootstrap loader. It loads core Java class extensions that are located in the jdk/jre/lib/ext/ path. Java extensions are optional class packages that augment the Java platform through their functions.
- Application Class Loader, also known as System ClassLoader, is a child of Extension ClassLoader. It loads application-specific classes from directories and jars specified by the CLASSPATH environment variable or the
java.class.pathsystem property, classes that programmers define for their specific application reside in that path.
The Delegation Model
What dictates which class loader loads a particular class is done through the Delegation hierarchy principle. This principle dictates that when a class loader receives a request to load a class, it first attempts to delegate the request to its parent. If the parent fails at loading this class, it then attempts to load it itself by searching for the .class file in its defined paths. The below diagram shows a high-level overview of the steps taken by the Class Loader Subsystem when receiving a request to load a class:
If the requested class is an application-specific class defined by the programmer, it will exist in the system classpath. The Application class loader initially receives the request. It delegates to the Extension class loader which in turn delegates to the Bootstrap class loader. The Bootstrap class loader fails to find the class in its paths and hence the responsibility is delegated back to the Extension class loader which fails as well. The responsibility then is handed back to the Application class loader which should find the class if it exists.
The ClassLoader class
The abstract ClassLoader class is extended to define particular class loaders in the system like the Extension class loader and the Application class loader. ClassLoader defines 4 main methods used in class loading:
Class defineClass(byte[] b, int off, int len): This method is core to class loading and is declared as final, a class file is first read into a byte array. The defineClass takes this byte array as an input and returns the equivalent Class object.Class findLoadedClass(String name): This method checks if a class is already loaded and returns its Class object or null if no class with that name is found.Class findClass(String name): This method attempts to locate a class’s class file from its defined paths and load its data (byte array). It then uses defineClass to create the equivalent Class object.Class loadClass(String name, boolean resolve): Any class loading starts with a call to this method. It uses all the above methods and enforces the delegation model. A skeleton of it is shown below:
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException
{
synchronized (getClassLoadingLock(name)) {
// check if the class has already been loaded
Class<?> c = findLoadedClass(name);
if (c == null) { //if class wasn't loaded before
// ...
try {
if (parent != null) {
//ask parent to load the class
c = parent.loadClass(name, false);
} else {
// if no direct parent ask BootstrapClassLoader to load
c = findBootstrapClassOrNull(name);
}
} catch (ClassNotFoundException e) {
// ClassNotFoundException thrown if
// parent fails to load class
}
if (c == null) { //parent failed to load class
//invoke own findClass to attempt to load the class
c = findClass(name);
// ...
}
}
return c;
}
}As seen in the above code,loadClass() first begins by checking if the class has been loaded before using findLoadedClass()(The ClassLoader class keeps track of previously loaded classes to prevent any confusion in the system that might arise if the same class was loaded multiple times). If the class wasn’t loaded before, it asks its parent class loader to load it by calling its parent’s loadClass(). If the parent fails to load the class (returns ClassNotFoundException), it calls its own findClass() to attempt to load the class. If findClass() fails to return the Class object then the method throws ClassNotFoundException.
As mentioned earlier, the class loading subsystem is exposed to programmers who can use it to load classes at runtime. The following is an example of a function that uses the Application Class Loader to load a new class at runtime:
public Class load (String name) throws ClassNotFoundException {
//get Application class loader object
ClassLoader classLoader = ClassLoader.getSystemClassLoader();
//load class using its name
Class cls = classLoader.loadClass(name);
return cls;
}Programmers can also extend ClassLoader and define their own class loaders which can actively participate in the class loading process. For example, they can define their own CustomClassLoader class and use it for custom class loading.
public class CustomClassLoader extends ClassLoader{ /*..*/ }ClassLoader cl = new CustomClassLoader(dir);
Class cls= cl.loadClass( "[class-name]" );They might do that for multiple reasons. Each class loader in the system has a set of defined paths through which it looks for class files. A developer might need to load class files from an unconventional location like an external database or repository and hence needs to define a specialized class loader that connects to and loads from these external locations.
Another reason could be to change some behavior in the class loading system. For example, the class loader system prevents loading a class more than once. A developer that needs to reload classes in their application might define a specialized class loader that overrides the default behavior and allows loading a class several times.
While there are no strictly set rules or best practices. Real uses of class loaders show the following patterns in overriding ClassLoader behavior:
findClass()is overridden when there’s a need to load classes from new locations.loadClass()is overridden when there’s a need to modify the delegation model or other default behavior in the class loading system.
The common requirement in all class loader use cases is the need to introduce new behavior to a program at runtime. The use of class loaders is usually tied with reflection as reflective programming enables programs to inspect and use newly introduced classes and hence integrate new types in the program. The next two practical examples are popular uses for class loaders, each needing to override ClassLoader behavior in a different way.
A Practical Example: Adding Plugin Functionality to a Java Application
Plugins are small programs that could be incorporated into a running application to extend it and provide new functionality. Consider a code editor like Visual Studio code, it aids development tasks like debugging, code testing, and version control. It also comes with an associated marketplace that has thousands of extensions that users can download. Extensions range from essentials like code editing for a certain programming language to luxury extensions like the Prettier, a code formatter used for a consistent code style (e.g. wrappers, indentation), and Better Comments extension which creates colored human-friendly code comments. Visual Studio code provides these extensions in a plug-and-play fashion.
While VS code isn’t written in Java, the idea behind plugin architecture is the same, the need for extending the system or plugging new functionality at runtime. Plugin functionality in a Java application is done with the aid of reflection and class loaders. Creating a simple plugin could be done in the following steps:
- Define a plugin Interface.
- Define plugin class loader.
- Define Protocol for installing and running plugins in the system.
For the first step, a unified interface should be defined through which the application can run any plugin.
public interface Plugin {
// returns the name of the plugin
public String getPluginName();
//runs the plugin functionality
public void run();
// used to configure plugin if needed
// returns true/false for success/failure to configure
public boolean configure(Object configuration);
}All plugins should have a class implementing this interface. This interface acts as a marker. When loading plugins, the application reflectively inspects the classes to search for the class that implements this interface. It contains the run() method which is the entry point for running the plugin. Alternatively, annotations could be used to mark a plugin’s run function similar to the JUnit example.
Secondly, a special class loader is defined for loading the plugin classes. This is desirable for many reasons:
- A class is uniquely identified by its name and class loader. Loading a plugin using a special class loader creates a unique namespace for plugin classes. This prevents conflicts from occurring between application classes and plugin classes. For example, if both the application and the plugin defined a class named Foo, the JVM will know how to distinguish between them since the application’s Foo class was loaded by the Application class loader while the plugin Foo class was loaded by a specialized class loader and each of them will have its own Class object.
- External plugin code should not be completely trusted for application security. It can contain untested, faulty, or malicious code and by having a separate class loader loading plugin classes, security restrictions on plugin classes can be enforced (this is out of topic for this post, but more about this here).
It would be neater to define a special path for all plugin code. The plugin class loader searches for class files in this path and could be defined as follows:
public class PluginClassLoader extends ClassLoader{
//path to folder containing plugin classes
String path;
public PluginClassLoader(String path){
/*call superclass construtor and setting PluginClassLoader's parent
to the Bootstrap class loader */
super(findBootstrapClassOrNull(name));
this.path=path;
}
public synchronized Class findClass(String name) throws ClassNotFoundException
{
// 1. gets class data
byte [] bytes = getClassData(name);
if ( bytes != null ){
//2. use defineClass to construct Class object
Class c = defineClass( null , bytes, 0, bytes.length );
return c;
}
//class data was not found, throw exception
throw new ClassNotFoundException();
}
//This is a helper method used to get the class data which is array of bytecode
public byte [] getClassData(String name){
try {
String classFile = classPath + name.replace('.', '/') + ".class";
int classSize = Long.valueOf((new File(classFile)).length()).intValue();
byte[] buf = new byte[classSize];
FileInputStream fileInput = new FileInputStream(classFile);
classSize = fileInput.read(buf);
fileInput.close();
return buf;
} catch(IOException e){
return null;
}
}
}The PluginClassLoader overrides findClass() which loads classes from the plugin path which is set in the path variable. (Note how PluginClassLoader does not override loadClass() since plugin class loaders shouldn’t interfere with the delegation model)
Finally, a protocol needs to be established for installing and running plugins, that is how the application adds, stores, and runs plugins and the steps taken by application users to install and run their custom plugins. Externally, installation steps can be:
- The user adds plugin code in a specified path.
- If the application has a command line, users can install and run plugins with specified commands. If the application has a user interface, designated buttons can be used for plugin installation and running
Internally, the application can define a PluginManager class that handles installing and running plugins (This class’s functions are linked to the plugin install/run commands or buttons). It can store all currently installed plugins in a hashtable containing (plugin name, Plugin object) pairs and define functions loadPlugin() and runPlugin() for loading and running plugins respectively. The following is an example of PluginManager:
public class PluginManager {
/* this map stores all the currently installed app plugins */
private Map<String, Plugin > plugins;
/* this variable stores the path which all plugin code exists */
String pluginsDir;
/* this function loads a plugin given the foldername, this folder
should exist in the plugin path and contains bundled plugin classes */
protected void loadPlugin(String foldername){
//initialize classloader
PluginClassLoader cl = new PluginClassLoader();
File dir = new File("[plugin-directory]");
String[] files = dir.list();
//loop over all classes in plugin directory
for (int i=0; i<files.length; i++) {
try {
//load class at current file
Class c = cl.loadClass(files[i].substring(0, files[i].indexOf(".")));
// get implemented interfaces
Class[] intf = c.getInterfaces();
/*loop over interfaces to discover if the class implements the
Plugin interface */
for (int j=0; j<intf.length; j++) {
/* if class implements Plugin interface */
if (intf[j].getName().equals("Plugin")) {
/* reflectively create an instance of this class */
Plugin plugin = (Plugin) c.newInstance();
/* save plugin in the plugins hashtable*/
String pluginName = plugin.getPluginName();
plugins.put(pluginName,plugin);
continue;
}
}
}catch (Exception ex) {
/*handle plugin loading exceptions*/
}
}
}
/* this function runs a specified plugin */
protected void runPlugin(String name) {
Plugin plugin = plugins.get(name);
plugin.run();
}
}As I mentioned earlier, every valid plugin should have a class implementing the Plugin interface. The loadPlugin() method creates an instance of PluginClassLoader and uses it to load all plugin classes from the specified plugin folder. It uses reflection to inspect plugin classes and discover the class that implements the Plugin interface and creates an instance of it. This instance is then stored in the plugins hashtable to run the plugin when needed. The runPlugin() method runs a specific plugin by getting its Plugin object from the plugins hashtable invoking the run() method.
The PluginManager class allows additional functionality to be added to a running application. It does this using class loaders and reflection. The next example is a more complex use of class loaders.
A Practical Example: Modifying Classes at Runtime
It's quite often that business requirements and operations change. Services or APIs provided by a business’s software can require modification down the line. Sometimes these modifications need to be done dynamically at runtime without a service restart. Since the unit of software in Java is a class, Java applications can dynamically update their logic by modifying their classes.
Dynamic class modification is a challenging problem and still an ongoing area of research in the Java world. There are several approaches to achieve this with some modifying the compiler and others modifying the JVM. However, users of the Java programming language can achieve the effect of class modification by carefully crafting their programs with the aid of class loaders and reflection. Class loaders can’t directly modify already loaded classes but they can reload the class file of an updated class. To achieve the class modification effect, the newly loaded class should replace the old class, this can be done using design patterns.
An active class is a loaded class that has instances in the JVM. Replacing an active class is a complex problem as there are many steps involved:
- Loading the new updated version of the class.
- Redirecting dependent parts of the application to now instantiate objects of the new class instead of the old.
- If instances of the old class exist in the system at that point of execution, they need to be replaced by equivalent instances of the new class.
For the first step in replacing a class, the updated class file must be loaded into the JVM. There is a nuance to this, which is that the ClassLoader class keeps track of all previously loaded classes by default and prevents loading the same class twice (check loadClass() implementation above). As mentioned earlier, a class in the JVM is identified by both its fully-qualified name (which contains the class’s name and the package which the class was loaded from like org.example.Foo ) and the class loader that loaded it. A class Foo loaded twice by two different class loaders will be seen as two different classes in the JVM (will have two different Class objects). There are two possible workarounds to reload a class Foo after it was modified:
- Change Foo’s fully qualified name by either changing the class’s name (e.g. FooV1, FooV2… etc.) or changing the package (within the classpath) in which its class file resides each time it's modified to trick the Application Class Loader into loading it again.
- Create a custom class loader and use it to load Foo multiple times. This involves overriding
loadClass()and programming it to always load certain classes even if they were loaded before. This is shown below:
public class CustomClassLoader extends ClassLoader{
String classPath ;
public CustomClassLoader(String path){
super(CustomClassLoader.class.getClassLoader());
this.classPath=path;
}
public synchronized Class findClass(String name) throws ClassNotFoundException{
// 1. gets class data
byte [] bytes = getClassData(name);
if ( bytes != null ){
//2. use defineClass to construct Class object
Class c = defineClass( null , bytes, 0, bytes.length );
return c;
}
//class data was not found, throw exception
throw new ClassNotFoundException();
}
//This method loads the byte code from the file using the classPath
public byte [] getClassData(String name){ ... }
@Override
public Class loadClass(String name) throws ClassNotFoundException {
// For a certian class or group of classes, always load
if(name.contains("Foo")) {
return this.findClass(name);
}
// for all other classes, use default load class implementation
return super.loadClass(name);
}
}CustomClassLoader overrides loadClass() and always loads a modifiable class Foo when requested to load it (it does this by calling itsfindClass() method which reads the most updated class file and returns the new Class object). This will allow the Foo class to be reloaded on demand. Notice that reloading Foo does not have any direct effect on the application. Here’s how the system looks after the Foo class is reloaded after some update to its class file:
Foo version 2 Class Object is not “seen” by the application and Foo version 1 is still in use and has active instances. There is no direct way to replace the Foo version 1 Class object with the Foo version 2 Class object. The next step is redirecting the application to use Foo Version 2. This is done using factory pattern and reflection.
Factory Pattern
Factory pattern is a creational pattern mainly used when object creation logic needs to be abstracted. This is done by creating a Factory class which is responsible for creating objects of a certain type. Internally this factory can be configured with the concrete types it should create with the desired rules. According to the classic Design Patterns: Elements of Reusable Object-Oriented Software book:
The key need for the factory pattern is that one object needs to create other objects but can’t know which particular concrete object to create beforehand
Rephrasing for our use case, a need for the factory pattern would be when one object needs to create other objects but can’t know which version of the object to create beforehand. For the aim of reloading classes, a modifiable class should have a Factory object. This object keeps track of the most updated version of a class by saving its Class object. It's also responsible for creating Foo objects and does so reflectively.
The first step in using the factory pattern is creating an interface for the modifiable class. This allows all dependent code to treat all the different versions of Foo the same.
public interface Foo {
public void doSomething();
}A modifiable class (ex. ConcreteFoo) implements the Foo interface. Next, a FooFactory is created as follows:
public class FooFactory {
//This variable stores the current implementing class object of Foo
static private Class implClass;
//This method returns an instance of the current implementing class
public static Foo newInstance() throws NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException {
Foo foo = (Foo) implClass.getDeclaredConstructor().newInstance();
return foo;
}
//This method reloads Foo class
public static void reload(String dir) throws ClassNotFoundException {
ClassLoader cl = new CustomClassLoader(dir);
implClass = cl.loadClass( "[ConcreteFoo]" );
}
}The FooFactory keeps track of the current Foo implementation using the implClass variable. It defines a newInstance() method which is a factory method as it “manufactures” a Foo object. It uses reflection to instantiate a Foo object using the current implementing class. The reload() method is called whenever the class is updated, it uses the CustomClassLoader to load the new Class object and then updates the implClass variable. This causes all subsequent calls to newInstance() to return instances of the updated version of the class.
Creating Foo objects across the application should happen exclusively through the FooFactory class to ensure consistency in the behavior of a Foo type. So far, the FooFactory ensures that all future uses of Foo will use the most updated class. The following code reloads a modifiable class and then creates a new instance of it using the FooFactory:
//Reload a modifiable class of type Foo after update
FooFactory.reload("[directory]");
//create new instance of Foo
Foo foo = FooFactory.newInstance();
foo.doSomething();Here’s how the system looks after the execution of the above code:
While definitely a step in the right direction since all subsequent usage of Foo will use Foo Version 2 (the updated class), old instances of Foo Version 1 still need to be replaced by equivalent instances of Foo Version 2 to complete the replacement.
An object exists in Java because it is referenced by some variable. Once an object is no longer used by any variable, it’s garbage-collected (deleted from the heap). If Foo objects exist at some point during execution, it's because some variables belonging to some objects are referencing them. For example, a class Bar might use Foo objects in its internal methods and hence, its references should ideally be updated to use the latest version of Foo for a full replacement effect to take place throughout the application.
Keeping track of all program objects that have internal references to Foo objects and updating their references would be a maintenance nightmare, especially if Foo objects are heavily used throughout the program or are used by other modifiable classes. Instead, the proxy pattern offers a neat solution. As a start, the Foo interface is updated as follows:
public interface Foo {
public void doSomething();
public Foo evolve(Foo foo);
}The added evolve() method specifies that each class implementing the Foo interface must define logic for creating an instance of itself that can act as a replacement for another Foo object. This method will be used to map objects of an old Foo implementation to objects from a new implementation.
Next Proxy pattern is used to replace old class instances.
Proxy Pattern
A Proxy in Java is an object which acts as a substitute for another object. It externally looks and acts like another object by implementing its interfaces but internally forwards its method calls to an actual instance of the object it imitates (its target), it might do extra processing before forwarding method calls to its target. Below is the sequence diagram of a proxy’s method call:
The key functionality achieved by the proxy pattern is that it allows method intercession. Interceding a method call means dynamically controlling the behavior that results from a method call. The proxy object is a middleman between the client code and a target object and can decide the resulting behavior from a method call. It can, for example, forward the method to a different target object or not forward the method call at all.
When reloading a class, running instances of the old version might still be used by other active classes in the application. For modifiable class Foo, Foo proxies can intercede method calls made by other application classes and forward the method call to the correct version of a Foo object. Consider the following:
- Instead of creating regular Foo objects through its factory method, The FooFactory returns a Foo proxy with its target set as an instance of the current implementing class.
- The FooFactory saves a reference to all proxies it creates in a list so that when a Foo class is reloaded, it loops over its list of proxies and updates each proxy’s target to an instance of the updated Foo class. It does this through the new class’s
evolve()method.
Using Foo proxies prevents other application classes from directly using Foo objects. This allows controlling the usage of the Foo class and simplifies interchanging Foo objects to simply setting the target of a proxy and not being concerned with the outside environment of the proxy and how Foo objects are being used in the system.
Factory pattern can be used to implement replaceable classes while Proxy pattern can be used to implement replaceable objects.
Before diving into the implementation. There are two important terminologies:
A proxy class is a class that implements an interface or a list of interfaces while a proxy instance is an instance of a proxy class.
Normally, to use the proxy pattern with a Foo type. Two classes need to be defined. The first is a legitimate class that implements the Foo interface while the second is the proxy class which also implements the Foo interface but has an internal target Foo instance to which it forwards its method its calls to.
Java’s reflection utilities (java.lang.reflect package) define Proxy API which automates the above steps. Java’s Proxy can dynamically create a proxy class given a list of interfaces and return proxy instances of it. The following is the partial definition of Proxy:
public class Proxy implements java.io.Serializable {
/* This method gets the proxy class of a type or set of types,
if it doesn't exist, one is created dynamically */
public static Class getProxyClass( ClassLoader loader,
Class[] interfaces )
throws IllegalArgumentException {...}
/* This method returns a proxy object of a specified array of interface types*/
public static Object newProxyInstance( ClassLoader loader,
Class[] interfaces,
InvocationHandler h )
throws IllegalArgumentException {...}
/* This method returns if a certian Class object is a proxy */
public static boolean isProxyClass( Class cl ) {...}
/* This method returns the invocation handler of a proxy object*/
public static InvocationHandler getInvocationHandler( Object proxy )
throws IllegalArgumentException {...}
}Without diving into much of the class’s details. The method that is mainly used in creating proxies is the newProxyInstance() method. This method returns a proxy object given a list of interfaces. It first checks if the input array of interfaces has a proxy class implementing them. If they don’t, one is created for them dynamically before returning an instance of it.
This method takes 3 arguments:
- The ClassLoader will be used to load the proxy class.
- An array of Class objects containing the list of interfaces that the proxy class implements.
- An InvocationHandler object that is responsible for handling requests received by the proxy instance. (more on this below)
The newProxyInstance() method simply creates a class that implements a set of interfaces to act as a proxy class and returns an instance of that class. Programmers still need to define what exactly should a proxy object do when receiving a method call. This is done through invocation handlers. An invocation handler is an object responsible for handling all method calls received by a proxy object. In other words, each proxy has an associated InvocationHandler object to which it forwards its requests.
The following is the InvocationHandler interface definition:
public interface InvocationHandler {
public Object invoke( Object proxy, Method method, Object[] args ) throws Throwable;
}A proxy object forwards its method calls to its InvocationHandler using the invoke() method. Its arguments are:
- A self-reference to the Proxy object.
- The Method Object of the invoked method on the proxy object.
- An array of Objects which are the arguments received by the proxy objects for the method call.
These inputs are all that an invocation handler needs to invoke a specific method. Developers create their custom InvocationHandler for their own use case. The following is the invocation handler defined for Foo:
public class FooIH implements InvocationHandler {
Foo target;
public FooIH(){
target= new ConcreteFoo();
}
public FooIH(Foo foo){
target = foo;
}
public void setTarget( Foo foo ){
target = foo;
}
public Foo getTarget(){
return target;
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
Object result = null;
result = method.invoke( target, args );
return result;
}
}The FooIH class has a Foo target variable which stores its target object. This object is an instance of the most updated version of a modifiable Foo class. Its invoke()method does not do any preprocessing but simply reflectively invokes the input method using its target object.
The following diagram shows all the objects involved when a proxy object method is invoked:
The proxy object forwards a method call to its InvocationHandler which in turn uses reflection to invoke the method on its target using the Method object of the invoked method.
Putting It All Together
This is the updated FooFactory class after adding proxy logic:
public class FooFactory {
static private Class implClass;
static ArrayList<WeakReference> proxies;
public static Foo newInstance() throws NoSuchMethodException, InvocationTargetException {
//create a Foo instance using the current implClass
Foo foo = (Foo) implClass.getDeclaredConstructor().newInstance();
//create a Foo proxy passing the foo object as the target
Foo fooProxy =(Foo) Proxy.newProxyInstance(
ClassLoader.getSystemClassLoader(),
new Class [] {Foo.class},
new FooIH(foo));
//add a reference to the new proxy in the proxies Arraylist
proxies.add(new WeakReference(fooProxy));
//return the proxy object
return fooProxy;
}
public static void reload(String dir) throws ClassNotFoundException, NoSuchMethodException{
//load new Class object using CustomClassLoader
ClassLoader cl = new CustomClassLoader(dir);
implClass = cl.loadClass( "ConcreteFoo" );
//get method "evolve" from the new class
Method evolve = implClass.getDeclaredMethod( "evolve", new Class[]{Object.class} );
ArrayList<WeakReference> updatedProxies = new ArrayList<>();
//loop over proxies arraylist
for ( int i = 0; i < proxies.size(); i++ ) {
//get current proxy
Proxy x =(Proxy)((WeakReference)proxies.get(i)).get();
if ( x != null ) {
//get proxy's invocation handler
FooIH fih = (FooIH)Proxy.getInvocationHandler(x);
//get target of invocation handler (this is the old object)
Foo oldObject = fih.getTarget();
//get the equivelant new Foo object through the evolve method
Foo replacement = (Foo) evolve.invoke( null, new Object[]{oldObject} );
//update the invocation handler's target to the new replacement object
fih.setTarget( replacement );
//add to references list
updatedProxies.add( new WeakReference( x ) );
}
}
proxies=updatedProxies;
}
}The following are the main updates in FooFactory:
- The added static ArrayList variable
proxiesstores a list of references to all proxy objects created by the factory. - The
newInstance()method now returns a Foo proxy instance instead of a regular Foo instance. The proxy is passed an instance of FooIH as its invocation handler (The FooIH object internally saves a target Foo object and forwards method calls to it). The method also now saves a reference of the created proxy in theproxiesarray list. - The
reload()method has an added task of looping over theproxiesarray list and updating their target to a Foo object from the updated class. For each proxy, it gets its target Foo object and creates an equivalent object of the new class using the new class’sevolve()method and sets it as the new target.
Note the usage of WeakReference object wrappers on the proxies list instead of storing a regular ArrayList of proxy objects.
static ArrayList<WeakReference> proxies;The reason behind this is that by directly storing references to Foo proxies, they are never garbage collected, even when they are no longer being used by the application. The WeakReference type allows referencing an object but does not prevent it from being garbage collected if it doesn’t have at least one strong reference. When an object is garbage collected, it’s WeakReference returns null.
The FooFactory now can completely replace a Foo Class and all its instances from the system. When reload() is invoked, an active modifiable Foo class will be transformed as follows:
This is indeed a complete class replacement! Note that after a while, the JVM will unload Foo Version 1 as it's no longer being used.
Conclusion
In this article, I explored Reflection and Class Loaders and their possible uses. Reflection allows dynamic typing and invocation while class loaders can dynamically load classes at runtime. They allow for flexible, configurable, and expandable systems. As useful as they are in some cases, there are a few things to keep in mind:
- Reflection comes with a performance overhead. Dynamic typing incurs more steps during execution than static typing. Moreover, reflection comes with security risks as many logic errors which are usually caught at compile time are pushed to runtime. This can lead to unexpected behavior in a program.
- Class loaders and reflection are considered antipatterns as they come in conflict with the more encapsulated systems the Java platform tries to bring. However, programming to interfaces and careful usage of design patterns can bridge the gap between the two, creating programs that use reflection when necessary while still maintaining OOP principles.
- Developers should avoid the trap of turning reflection into an overused Golden Hammer technique. Overly engineered systems that are excessively flexible and configurable aren’t necessarily maintainable. After all, this is the OOP territory. Modularity and encapsulation are encouraged because they produce robust maintainable systems. Reflection is more of a cheat code for necessity rather than a way to design applications in Java.
Note: I’m working on a mini-application that serves as a full example that provides plugin functionality and modifiable classes. I will post its GitHub link here once finished.
References:
- Java's official documentation
- Java Reflection in Action By Ira R Foreman, Nate Foreman
- The Well-Grounded Java Developer by Benjamin Evans, Jason Clark, and Martijn Verburg
