I worked 6 months at WSO2 as an Software Engineering intern from November 2013. That period of 6 months is undoubtedly the best ever period of time I had spend as a university student so far. There were 24 students from our batch with me in training. Other than the great experience working with an open source community, we had a great time filled with entertainment.
After we finished the intern project(The first phase of the training), I and two of interns from UoM were assigned to WSO2 API Manager Team. I had a great time with them filled with team work and got a lot of experience. It was when I had a chance to develop this new feature to WSO2 API Manager Product, "Introducing Default API Version".
Introducing a Default Version to API
curl -H "Authorization :Bearer <token>" http://<host>:<port>/<apiname>/<apiversion>/<resource>
By introducing this feature a subscriber can invoke the same API like this without the version field.
curl -H "Authorization :Bearer <token>" http://<host>:<port>/<apiname>/<resource>
By this feature, API provider is given the facility to select any of the APIs any time as Default Version.
By introducing this feature a subscriber can invoke the same API like this without the version field.
curl -H "Authorization :Bearer <token>" http://<host>:<port>/<apiname>/<resource>
By this feature, API provider is given the facility to select any of the APIs any time as Default Version.
Benefits for the API Providers and Subscribers with this feature:
APIs change time to time like everything. May be there are new features introduced, bugs are resolved, improved the performance and etc. Then usually what happens is, the provider need to introduce another API version to the API Store. Then the subscribers of same API see that in the API store and need to be resubscribed. By introducing this feature, the provider can easily avoid that by selecting the Latest Stable version as the Default API. When there is such an occasion to introduce a new API, the provider can select it as the Default version. Then the current subscribers easily are switched to the new API without re-subscription. This avoids the delay occurring with re-subscription period.
With this feature, the subscribers of the Default API Version can easily be switched to another API with a very small effort of the provider side.
With this feature, the subscribers of the Default API Version can easily be switched to another API with a very small effort of the provider side.
Implementation
High level implementation can be summarized by this diagram.
By this new implementation, a new API proxy is introduced to the Synapse Gateway. When defining an API to the gateway usually the version is specified but for this API it is not. It basically does the relevant routing the configured API as default when it receives a request.
We further discussed about the architecture of the solution and the following was considered.
- Introduce a new attribute to the API artifact to indicate whether an API is Default or not
In API Manager, registry is used as the storage for the meta data and resources. Accordingly the currently created APIs' meta data are stored there. For the new feature a new field is introduced for the api.rxt template to hold the default API version attribute.
- Creating a separate table in the API Manager database to keep the information of APIs which are selected as default.
Above was a sample h2 database table which is used to store Default API related information. There are two default API related columns which are DEFAULT_API_VERSION and PUBLISHED_DEFAULT_API_VERSION. According to the API life-cycle the default version API can also be in those states. So DEFAULT_API_VERSION field is used to hold the actual version that the provider selected as default and the PUBLISHED_DEFAULT_API_VERSION is used to hold the API version that is in published state. According to that the default API can be in following states.
There were two reasons behind maintaining two such fields. In this feature, the provider can change default API version anytime(Default version can be switched to another version any time. But in that occasions we have to make sure that the existing users do not break after changing the Default API. So once the subscribers are using one API as Default there is a possibility that the provider select an API as default which is in CREATED state. So now all the Apps which are using the default API will break. To avoid that a separate column is maintained to store the published version of the Default API. In such occasions the apps are temporary routed to the previous default version until the current Default API is taken to the PUBLISHED state.
The other reason is that currently the apps are authenticated at the gateway when they are invoking an API. For that, a token which is generated through the Thrift server is used. At the gateway, there is a handler called Authentication Handler and it does that validation. The problem which arise when changing the Default Version API is that the users are now routed to a new API and the users may or may not be subscribed to that API. So if the normal authentication flow applied, most probably the validation will fail and the app will not be allowed to call the new API. So a slight change is done to the key validation process such a way that if a user is previously subscribed to the Default API and now accessing the Published version of the Default API(which is stored in PUBLISHED_DEFAULT_API_VERSION field) then he is allowed to invoke the Default API. Then it make sure that the existing user does not break after changing the Default API.
The following State diagram briefs the above states of the Default API.
When it is needed to add an API to the gateway an API proxy should be defined in the gateway. Currently Synapse gateway is used(Actually Apache Synapse is the core of the WSO2 ESB which is the gateway of API Manager). The API proxy is defined using XML.
A sample API proxy which is published in the gateway uses the following pattern in XML.
<api name="provider1--youtube" context="/youtube">
<resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
<inSequence>
<!-- Processed when a request received from the client -->
<send>
<endpoint>
<!-- defines the back end endpoints for the API -->
<http uri-template="http://gdata.y">
<timeout>
<duration>30000</duration>
<responseAction>fault</responseAction>
</timeout>
…..
…..
</http>
</endpoint>
</send>
</inSequence>
<outSequence>
<!-- Processed when a response received from the backend -->
<send/>
</outSequence>
</resource>
<handlers>
<handler>
<!-- defines handles for the API to record statistics etc..-->
<handler class="org.wso2.carbon.apimgt.gateway.handlers.common.SynapsePropertiesHandler"/>
</handler>
</handlers>
</api>
<resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
<inSequence>
<!-- Processed when a request received from the client -->
<send>
<endpoint>
<!-- defines the back end endpoints for the API -->
<http uri-template="http://gdata.y">
<timeout>
<duration>30000</duration>
<responseAction>fault</responseAction>
</timeout>
…..
…..
</http>
</endpoint>
</send>
</inSequence>
<outSequence>
<!-- Processed when a response received from the backend -->
<send/>
</outSequence>
</resource>
<handlers>
<handler>
<!-- defines handles for the API to record statistics etc..-->
<handler class="org.wso2.carbon.apimgt.gateway.handlers.common.SynapsePropertiesHandler"/>
</handler>
</handlers>
</api>
The same sample is followed when defining a Default API proxy configuration to the gateway. This is the sample API that is used to define a Default API in the gateway.
<api name="provider1--youtube" context="/youtube">
<resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
<inSequence>
<property name="isDefault" expression="get-property('transport', 'WSO2_AM_API_DEFAULT_VERSION')"/>
<filter source="get-property('isDefault')" regex="true">
<then>
<log level="custom">
<property name="STATUS" value="Faulty invoking through default API.Dropping message to avoid recursion.."/>
</log>
<payloadFactory media-type="xml">
<format>
<am:fault xmlns:am="http://wso2.org/apimanager">
<am:code>500</am:code>
<am:description>Faulty invoking through default API</am:description>
</am:fault>
</format>
<args/>
</payloadFactory>
<property name="HTTP_SC" value="500" scope="axis2"/>
<property name="RESPONSE" value="true"/>
<send/>
</then>
<else>
<header name="WSO2_AM_API_DEFAULT_VERSION" scope="transport" value="true"/>
<property name="uri.var.portnum" expression="get-property('http.nio.port')"/>
<send>
<endpoint>
<http uri-template="http://localhost:{uri.var.portnum}/youtube/2.0">
<timeout>
<duration>30000</duration>
<responseAction>fault</responseAction>
</timeout>
</http>
</endpoint>
</send>
</else>
</filter>
</inSequence>
<outSequence>
<send/>
</outSequence>
</resource>
<handlers>
<handler class = "org.wso2.carbon.apimgt.gateway.handlers.common.SynapsePropertiesHandler"/>
</handlers>
</api>
<resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
<inSequence>
<property name="isDefault" expression="get-property('transport', 'WSO2_AM_API_DEFAULT_VERSION')"/>
<filter source="get-property('isDefault')" regex="true">
<then>
<log level="custom">
<property name="STATUS" value="Faulty invoking through default API.Dropping message to avoid recursion.."/>
</log>
<payloadFactory media-type="xml">
<format>
<am:fault xmlns:am="http://wso2.org/apimanager">
<am:code>500</am:code>
<am:description>Faulty invoking through default API</am:description>
</am:fault>
</format>
<args/>
</payloadFactory>
<property name="HTTP_SC" value="500" scope="axis2"/>
<property name="RESPONSE" value="true"/>
<send/>
</then>
<else>
<header name="WSO2_AM_API_DEFAULT_VERSION" scope="transport" value="true"/>
<property name="uri.var.portnum" expression="get-property('http.nio.port')"/>
<send>
<endpoint>
<http uri-template="http://localhost:{uri.var.portnum}/youtube/2.0">
<timeout>
<duration>30000</duration>
<responseAction>fault</responseAction>
</timeout>
</http>
</endpoint>
</send>
</else>
</filter>
</inSequence>
<outSequence>
<send/>
</outSequence>
</resource>
<handlers>
<handler class = "org.wso2.carbon.apimgt.gateway.handlers.common.SynapsePropertiesHandler"/>
</handlers>
</api>
First part of the definition is done as a fail-safe operation. I had fixed a bug related to synapse APIs. But if anything went wrong or the patch is not properly applied there was a big possibility to message received to the Default API is being blocked and recursively processed. So to avoid that before sending the message out, a header called WSO2_AM_API_DEFAULT_VERSION is set. Then if the same message circulated to the Default API again, the header will be there and it can be easily identified that the message has been circulated. So if a message is received, it will check for the above header and if it is there, the message will be dropped with a server log and sending a fault message to the client.
I used a separate handler to the above API which is SynapsePropertiesHandler. When forwarding to the relavent API the Default API should know which port the synapse is up. Hard coding it can cause problems in the future if the port of the server was changed after creating several number of APIs, because the port mentioned in the proxy configuration is not changed. So the messages will not be sent to the API. The best way to do is taking that value from the synapse message context. The handler fetches that value on runtime and sets to a property called {http.nio.port}. That value is taken as the port when forwarding the messages.
Following is the implementation of that simple handler class. When whiting a handler the class should be extended from the synapse AbstractHandler class.
public class SynapsePropertiesHandler extends AbstractHandler{
public boolean handleRequest(MessageContext messageContext) {
String httpport = System.getProperty("http.nio.port");
messageContext.setProperty("http.nio.port", httpport);
return true;
}
public boolean handleResponse(MessageContext messageContext) {
return true;
}
}
public boolean handleRequest(MessageContext messageContext) {
String httpport = System.getProperty("http.nio.port");
messageContext.setProperty("http.nio.port", httpport);
return true;
}
public boolean handleResponse(MessageContext messageContext) {
return true;
}
}
Fields of the above API proxy can be changed according to the API which is being set as Default. Those parameters are API name, provider name, version to be routed the message. So when producing the above proxy definition certain fields should be taken care as variables.
When producing existing API configurations API Manager uses Apache Velocity. Apache Velocity is a Java-based template engine. It primarily permits web page designers to reference methods defined in Java code. Other than developing web pages Velocity can be used to generate SQL, PostScript and other output from templates. It can be used either as a standalone utility for generating source code and reports, or as an integrated component of other systems.
In Velocity VTL(Velocity Template Language) is used. Inside the template we can define variables so that they can be set as Java objects during processing the template. Following is the brief of the template in VTL I used as the Default API.
<api xmlns="http://ws.apache.org/ns/synapse" name="$!apiName" context="$! apiContext">
<resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
<inSequence>
<property name="isDefault" expression="get-property('transport', 'WSO2_AM_API_DEFAULT_VERSION')"/>
<filter source="get-property('isDefault')" regex="true">
<then>
...
<send/>
</then>
<else>
...
<send>
<endpoint>
<http uri-template="http://localhost: {uri.var.portnum}/$!{fwdApiContext}/$!{defaultVersion}">
...
</http>
</endpoint>
</send>
</else>
</filter>
</inSequence>
<outSequence>
<send/>
</outSequence>
</resource>
<handlers>
<handler …/>
</handlers>
</api>
<resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
<inSequence>
<property name="isDefault" expression="get-property('transport', 'WSO2_AM_API_DEFAULT_VERSION')"/>
<filter source="get-property('isDefault')" regex="true">
<then>
...
<send/>
</then>
<else>
...
<send>
<endpoint>
<http uri-template="http://localhost: {uri.var.portnum}/$!{fwdApiContext}/$!{defaultVersion}">
...
</http>
</endpoint>
</send>
</else>
</filter>
</inSequence>
<outSequence>
<send/>
</outSequence>
</resource>
<handlers>
<handler …/>
</handlers>
</api>
public String getConfigStringForDefaultAPITemplate(String defaultVersion) throws APITemplateException {
StringWriter writer = new StringWriter();
try {
VelocityEngine velocityengine = new VelocityEngine();
velocityengine.init();
ConfigContext configcontext= new APIConfigContext(this.api);
VelocityContext context = configcontext.getContext();
context.put("defaultVersion",defaultVersion);
String fwdApiContext=this.api.getContext();
if(fwdApiContext!=null && fwdApiContext.charAt(0) == '/')
fwdApiContext=fwdApiContext.substring(1);
context.put("fwdApiContext",fwdApiContext);
Template t = velocityengine.getTemplate(this.getDefaultAPITemplatePath());
t.merge(context, writer);
} catch (Exception e) {
log.error("Velocity Error", e);
throw new APITemplateException("Velocity Error", e);
}
return writer.toString();
}
StringWriter writer = new StringWriter();
try {
VelocityEngine velocityengine = new VelocityEngine();
velocityengine.init();
ConfigContext configcontext= new APIConfigContext(this.api);
VelocityContext context = configcontext.getContext();
context.put("defaultVersion",defaultVersion);
String fwdApiContext=this.api.getContext();
if(fwdApiContext!=null && fwdApiContext.charAt(0) == '/')
fwdApiContext=fwdApiContext.substring(1);
context.put("fwdApiContext",fwdApiContext);
Template t = velocityengine.getTemplate(this.getDefaultAPITemplatePath());
t.merge(context, writer);
} catch (Exception e) {
log.error("Velocity Error", e);
throw new APITemplateException("Velocity Error", e);
}
return writer.toString();
}
VelocityContext instance is used to share the references to build the outcome using the template. This implementation uses a HashMap (java.util.HashMap ) for data storage. To generate the outcome Context and the Template is merged.
After the required configurations were generated they should be published in the gateway. The API Manager gateway such that several gateways can exist. In such occasions the configuration should be copied to all the other instances as well.
