Sunday, December 20, 2009

Spring 3.0 GA

Spring 3.0 is now GA. Spring has already fully demonstrated to be a successful alternative to EJB development. JEE 6 has learnt from Spring and simplified a lot enterprise Java development, becoming much more lightweight and usable. It shows that some good degree of competition can improve the market, but I hope Spring will keep following its own philosophy instead of trying too much to compete with JEE. the risk I see is that Spring could become too wide and difficult to understand.
Spring has demonstrated that the open community can produce better products than vendors' committees, so keep doing the good job of simplifying enterprise development.

Friday, December 11, 2009

Competitive Strategies

If you want to learn and discuss about competitive strategy, my friend Marco has started a very interesting blog on the subject. As an expert manager and consultant, Marco collects is thoughts and discuss books and articles about strategic management. Sometime we need to stop, exit from day by day activities and think out of the box to evaluate whether we are following a strategic path or just following short-term tactics.

Somebody once wrote: Strategy without tactic is a dream. Tactic without strategy is a nightmare.

Tuesday, November 3, 2009

Fuji Simple Service API, SpringDM and OSGi Integration with JBI

Sujit Biswas wrote an interesting article which:

"[...] shows how to integrate Fuji simple service api with spring/springDM application, We will create a springDM application which instantiates the above provider Implementation , based on spring-config and spring-osgi-config files."

Fuji allows for Spring and OSGi services integration

"The simple service provides the capability to invoke/interact with JAVA code outside of JBI, i.e Service API provides a bridge between code that is hosted outside of a JBI component and service endpoints available on the NMR."

This enables integration of OSGI/POJO services with JBI. Read Sujit's article for more.

About Fuji:

"Project Fuji forms the core component of Open ESB v3 effort and represents Sun's next generation open source integration runtime, focused on providing a lightweight, developer-friendly, and extensible platform for composite application development.

At the core of Project Fuji is a lightweight micro-kernel based on JBI (JSR 208) and OSGi. Packaged as an OSGi bundle, the micro-kernel can be installed in any OSGi R4 compliant runtime (such as Felix, Knopflerfish or Equinox), including GlassFish v3. JSR 208 support introduces a robust, message-based service bus to the OSGi environment and allows the wide range of existing JBI "components" (adapters and containers) to run in Fuji.

Developer experience is a primary focal point in Fuji as evidenced by the level of flexibility and accessibility offered in the platform. Starting with a rapid, top-down development language, IFL (Integration Flow Language), developers can quickly and easily generate an integration application using a domain-specific grammar. The service development model favors convention and configuration over boilerplate code and framework APIs, allowing integration developers to focus on the code that matters. "

Wednesday, October 7, 2009

Java CAPS 6.2 to be released

There are rumours that the latest Java CAPS 6.2 will be released within end of this month. So far Sun has already published the release notes.

In summary, from the release notes:

This release of Java CAPS uses the following versions of NetBeans and GlassFish:

  • NetBeans IDE 6.5.1
  • GlassFish Enterprise Server 2.1 patch 2

This release of Java CAPS provides new support for the following operating systems:

Design-Time

  • Windows Server 2008
  • Red Hat Enterprise Linux AS 5.3 (64 bit)

Runtime

  • Windows Server 2008 (OS: 32 bit; JVM: 32 bit)
  • OpenSolaris 2008.11 (OS: 32 bit; JVM: 32 bit)
  • Red Hat Enterprise Linux Server 5.3 (OS: 32 bit; JVM: 32 bit)
  • Red Hat Enterprise Linux Server 5.3 (OS: 64 bit; JVM: 32 bit)
  • IBM AIX 5L 5.3 (OS: 64 bit; JVM: 32 bit)

This release of Java CAPS adds support for the following browsers and external systems:

  • Firefox 3.5.2 for accessing all Java CAPS browser components
  • OpenSSO Web Services Security (WSS) Agent for the HTTP Adapter
  • SWIFT 2009 messages and SWIFT 2009 certification
  • SAP JCo 3 for the SAP BAPI Adapte
My comments:
  • It is a bit disappointing that the IDE is not Netbeans 6.7. I have experienced JAX-WS Web Services deployment issues with Netbeans 6.5 and the suggested solution has been to upgrade to Nb 6.7 (!?!) so I really hope Sun has fixed some bugs in Nb 6.5 before packaging it with JCAPS. It would be annoying to start patching the stuff the day after downloading it.
  • Finally Java CAPS has an understandable version numbering convention: this release will be called 6.2 and not "6 update 2", which meant nothing unless you want to remember customers a sound close to the Windows service packs...

Monday, September 28, 2009

Tungsten

It seems there is a fork of the old C-JDBC project that I was not aware of:

Tungsten is an open source stack for deploying and operating horizontally scaled database clusters, also known as database scale-out. Scale-out works by spreading data across multiple, independent database servers connected through a network. The model offers an intuitive, incremental approach to solving the following important database problems:

  • Preventing data loss though up-to-date replica databases and coordinated backups
  • Increasing overall application availability by providing rapid database failover
  • Raising performance and through-put by dispatching read traffic to replicas
  • Integrating data between heterogenous systems, for example to support scaling of commercial databases using low-cost open source databases.
C-JDBC is:

an open source (LGPL) database cluster middleware that allows any Java application (standalone application, servlet or EJB container, ...) to transparently access a cluster of databases through JDBC(tm). The database is distributed and replicated among several nodes and C-JDBC balances the queries among these nodes. C-JDBC handles node failures and provides support for checkpointing and hot recovery.


Saturday, June 27, 2009

EJB 3.1 specs heading in the right direction

Since a long time EJB were loosing ground in favour of simpler and more effective programming models. specifically, Spring has become the leading framework to develop and deploy JEE applications.


For these reasons, a sense of urgency is pushing the new EJB specifications toward a massive simplification. This trend was clearly significant when the EJB 3.0 were introduced, and it is continuing now with the latest EJB 3.1.

Ken Saks, the Specification Lead for EJB 3.1, has written a nice summary of the hottest features. "A Sampling of EJB 3.1" shows you the most interesting news. Together with the upcoming Glassfish V3 , which is a very lightweight and OSGi based application server, the new specs can represent a quantum leap in terms of productivity and win back some developers.

It is difficult to predict if it is too late to regain developers' confidence and market quotas, but the combination of JEE 6 + EJB 3.1 + Glassfish V3 seems to be heading in the right direction: no overload of features but simplification.

Specifically, Ken Saks mentions:
  • Ease of Development: the ability to package enterprise bean classes directly in a .war file, without the need for an ejb-jar
  • No-interface View: you don't have to write interfaces if you don't need it
  • Simplified Packaging: You now have the option of placing EJB classes directly in the .war file, using the same packaging guidelines that apply to web application classes.
  • Singletons: A singleton is a new kind of session bean that is guaranteed to be instantiated once for an application in a particular Java Virtual Machine
  • Application Startup/Shutdown Callbacks: The introduction of singletons also provides a convenient way for EJB applications to receive callbacks during application initialization or shutdown

Simplification, is the way to go.

Monday, June 22, 2009

JMS Over HTTP Using OpenMQ

DZone has an interesting article about new OpenMQ's features:

"OpenMQ provide different transport protocol and access channels to access OpenMQ functionalities from different prgramming . One of the access channels which involve HTTP is named Universal Message Service (UMS) which provide a simple REST interaction template to place messages and consume them from any programming language and device which can interact with a network server. UMS has some limitations which is simply understandable based on the RESTful nature of UMS. For current version of OpenMQ, it only supports Message Queues as destinations so there is no publish-subscribe functionality available."

Read the full article.
About OpenMQ.

Thursday, June 18, 2009

GlassFish ESB v2.1 has been released

"New in GlassFish ESB v2.1 is clustering for all components, a great number of bug fixes, the inclusion of the IEP SE and Scheduler BC (a new component!), several component enhancements, inclusion of the latest NetBeans 6.5.1 IDE and the latest GlassFish v2.1 application server, and support for AIX 5.3. "

Look at the release notes for more.

Wednesday, April 15, 2009

How to call a remote transactional Weblogic EJB from a Java client

My objective here is to show how to call a remote Weblogic stateless EJB from a Java client, then adding the ability to invoke multiple EJBs in a client-controlled transaction.

The code below is very simple and has been tested with BEA Weblogic Application Server 10.0 and developed with the Netbeans 6.1 IDE. The original requirement was to call Weblogic EJBs from Glassfish, using the Weblogic user transactions from the client code. Let's say there is the need to call several EJBs and to make it in a single unit of work, this is not usually a best practice, as client-controlled transactions are usually slower and lead to bad design. The best would be to expose a transactional service from Weblogic which wraps all the calls in one server-side unit of work, but it is not always feasible if we are not in control of the other side and this is also an example how to call a remote EJB from Java, if one removes the transactional code.

Anyway, let's have a look at our sample JEE 5 service (EJB 3), which is deployed in Weblogic:
package com.mt.ejb;
import javax.ejb.Stateless;

@Stateless(mappedName = "CalculatorBean")
public class CalculatorBean implements CalculatorRemote {
        public int add(int a, int b) {
        return a + b;
    }
}



Above is a trivial "hello world" EJB, implementing a Remote interface. The mappedName = "CalculatorBean" is a mandatory parameter for the @Stateless annotation if one wants the EJB to be remotely callable.

I deployed in Weblogic, Netbeans offers the ability to connect to a Weblogic server, even if the plugin has limited functionalities it gives me what I need and allows me to work from my favourite IDE and deploy from there.


Here comes the client code, which is more interesting:

package com.mt.client;

import com.mt.ejb.CalculatorRemote;
import java.util.Hashtable;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.transaction.UserTransaction;

public class WeblogicTest {

    private final Logger log = Logger.getLogger(WeblogicTest.class.getName());

    public WeblogicTest() {
    }

    public void callRemoteEJB() {
        InitialContext context = null;
        UserTransaction transaction = null;

        Hashtable env = new Hashtable();
        env.put(Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
        env.put(Context.PROVIDER_URL, "t3://localhost:7001");
        env.put(Context.SECURITY_PRINCIPAL, "weblogic");
        env.put(Context.SECURITY_CREDENTIALS, "weblogic");

        try {
            context = new InitialContext(env);
            transaction = (UserTransaction) context.lookup("javax.transaction.UserTransaction");
            if (transaction != null) {
                log.info("in transaction context.");
                try {
                    CalculatorRemote calculator = (CalculatorRemote) context.lookup("CalculatorBean#com.mt.ejb.CalculatorRemote");
                    transaction.begin();
                    int result = calculator.add(3, 5);
                    transaction.commit();
                    log.info("add=" + result);
                } catch (Exception ex) {
                    ex.printStackTrace();
                    try {
                        transaction.rollback();
                    } catch (Exception ex1) {
                        log.log(Level.SEVERE, "Can't rollback transaction.", ex1);
                    }
                }
            } else {
                log.info("transaction context is null.");
            }
        } catch (NamingException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        WeblogicTest test = new WeblogicTest();
        test.callRemoteEJB();
    }
}

It is necessary to add the CalculatorBean EJB jar file and the wlclient.jar to build the above client code. The wlclient.jar can be usually found into folder \server\lib of our Weblogic server.


Let's comment the main code sections.

First, I populate the connection properties:
Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
env.put(Context.PROVIDER_URL, "t3://localhost:7001");
env.put(Context.SECURITY_PRINCIPAL, "weblogic");
env.put(Context.SECURITY_CREDENTIALS, "weblogic");
I look-up the initial context and get the UserTransaction from Weblogic's JTS:
context = new InitialContext(env);
transaction = (UserTransaction) context.lookup("javax.transaction.UserTransaction");
I lookup and call the remote EJB by enclosing the method invocation in transaction:
CalculatorRemote calculator = (CalculatorRemote) context.lookup("CalculatorBean#com.mt.ejb.CalculatorRemote");
transaction.begin();
int result = calculator.add(3, 5);
transaction.commit();
The lookup string "CalculatorBean#com.mt.ejb.CalculatorRemote" makes the trick, it is the Weblogic syntax to identify the Bean and its Remote interface.

Of course, in this trivial case there are not any transactions, but I wanted to keep the example very simple. If the EJB was transactional, the call made use of the transactional context, while in this case it is simply ignored.

Friday, April 3, 2009

Farewell SGI

The glorious SGI, after being close to bankruptcy and going for chapter 11, has been bought by Rackable for mere $25 millions plus the assurance to pay its debts. I learned 3D computer graphics and OpenGL programming in the late 90s on SGI's Indigo and Octane workstations, beautiful pieces of hardware and software but already feeling the price pressure coming from much cheaper Windows workstations. It is a piece of the Silicon Valley computing history which was hit by the post New Economy bubble crisis and has never really recovered from that point.
Farewell, SGI.

Tuesday, March 10, 2009

The Crisis of Credit Visualized

The Crisis of Credit Visualized by Jonathan Jarvis, an interaction and media designer, is one of the best explanation of the present credit crisis I have found on the Internet. It also a wonderful example on how to create a beautiful visual presentation.



Saturday, March 7, 2009

GlassFish ESB Roadmap

The OpenESB community just released a roadmap for this year. Things are accelerating.

GlassFish ESB v2.1


Scope:

* NetBeans 6.5
* GlassFish 2.1
* Full support for GlassFish clustering
* Scheduler BC
* Misc. performance enhancements
* Bug fixes

Target date: Mid May 2009


GlassFish ESB v3

Scope:

* NetBeans 6.7
* GlassFish 3
* Light weight composition
* IFL
* Maven Support
* OSGi
* Interceptors
* ...

Target date: Sept 2009

So the most interesting will be GF ESB v3 scheduled for September 2009. Glassfish V3 is a micro-kernel based app server running on a OGSi platform. The IFL language is an effort to simplify common ESB tasks and implementing a domain specific language, much like Apache Camel, to implement common enterprise integration patterns and messages manipulation (routing, filtering, splitting, pipelining, ...).

Friday, March 6, 2009

The OpenESB and Java CAPS consulting ecosystem

Sun's new open-source strategy offers interesting consulting opportunities. In the last months I have worked closely with former SeeBeyond and Sun colleagues to build a network of partners able to independently support the free OpenESB and commercial Java CAPS platforms at all levels. A set of very skilled professional left Sun recently and joined the freelance market or started new companies. Most of these expert people were actually also in charge of developing core aspects of the Glassfish ESB, Java CAPS and MDM suites, so it is possible to even offer product's enhancements and customizations. Sun is supporting this effort, so we are more and more closely coordinating activities with Sun Services in US and EMEA.

It is a huge paradigm shift from SeeBeyond. When I was in SeeBeyond everything was closed and even secret, while Sun is now growing as an open company embracing open-source as a way to tackle the market from a very different angle than its competitors. At the beginning many Sun employees were doubtful about new Sun strategy, it is so distant from the company's culture. Sun was, and partially still is, an hardware and infrastructural software vendor, a company built by engineers for engineers, with a technology language and culture. Integration and SOA are much more related to pure business aspects, customer's expectation are very different because they take for granted that the technological problems are solved, what they want is business integration, which requires a lot of consulting expertise. This kind of combination of technical and business expertise is exactly in the DNA of former SeeBeyond consultants.

The network of partners is growing and able to fill the business gap. SOA is not much related to technologies, it is about designing and fitting an Enterprise Architecture, it requires first to understand customer's business needs. The OpenESB ecosystem is made of a set of commercially supported open-source tools which provides the possible best value for money. Now there exists also a worldwide consulting services offering able to put this software effectively in production.

OpenESB is an open community, where Sun, its partners and customer can collaborate to create the best set of features, pro-actively suggesting functionalities and corrections. In which ways OpenESB offering is different from both big vendors and other similar open-source products?
  • It is fully open-source and supporting open standards, so you can really avoid any vendor lock-in
  • Full commercial support and indemnification is optionally provided by Sun at much cheaper prices than any other big vendor
  • You can download the suite, develop your solution and then decide if and when buy commercial support
  • Comparing to other open-source ESB, support is provided by a much bigger company, the one who invented Java
  • The set of development and runtime environments (Netbeans, Glassfish, OpenMQ, JBI) is very mature and allows for better developers productivity than other open-source ESB
Netbeans make a big difference: this is a great IDE and the company behind it is the same building OpenESB, so you get a closer integration between the IDE and the ESB in the whole development cycle. Other products require you to edit by hand lot of configuration files, IDE integration is weak and often provided by third parties, unless you decide to buy a much more expensive commercial ESB from any other big vendor. Glassfish V2 is now a great JEE 5 application server, Glassfish V3 is going to be the market leader in terms of features.

Glassfish ESB IDE, based on Netbeans 6.1
So, what are the real services a customer can expect from the network of professional OpenESB partners?
  • High level consulting expertise, most consultants are former SeeBeyond and Sun professional services. No kids, senior people only with up to 25 years experience
  • Possibility to leverage both Sun's technical support and the open-source community, to get independent advices
  • Ability to modify or create custom JBI adapters, as many companies in the network are OpenESB code committers and some were even Sun product managers (can you really ask your Big Vendor to create a custom adapter for you?)
  • For previous SeeBeyond customers there is also a lot of added value, as the network can offer tailored migration plans from DataGate, eGate 4.x, ICAN 5.0, JCAPS 5.1 to OpenESB and Java CAPS 6
  • Ability to support big enterprises worldwide. I'm in touch with partners from Israel to Scandinavia, and in US also. We are looking for partners in Australia and New Zealand shortly.
  • Any vertical market: retail, healthcare, telecommunications, finance, energy, ...
  • Any EAI experience: SAP, Siebel, Oracle, CICS, SWIFT, Microsoft, ...
  • Identity management, Role management & provisioning, Security, Master Data Indexes, federated SSO, SOA Governance, Business Process Management, Rule Engines.
  • On going partnership discussions with Savvion, Fair Isaac, Autonomy and other specialized vendors.
An independent and comprehensive review of Glassfish ESB can be read here.

For more information about the "unit-42" partnership network, please drop an email to me or my business partner.

Tuesday, February 10, 2009

New Sun Glassfish Portfolio

Sun has announced the new Glassfish Portfolio.

Finally Sun seems is learning something from MySQL, they are selling added-value tools for paying customers.

"GlassFish Enterprise Server subscribers are entitled to Enterprise Manager, which delivers improved management and performance monitoring features."

Before the Enterprise Manager was available only for Java CAPS. The new version seems to be a lot improved anyway:
  • SNMP monitoring: Leverages existing SNMP infrastructure to monitor GlassFish
  • Performance Advisor: Proactively monitors key performance indicators and automatically tunes JDBC connection pools.
  • Performance Monitor: Diagnoses performance bottlenecks with visibility into key GlassFish performance metrics.
This was a much needed tool for enterprise customers, as monitoring and management is one of the primary features of any ESB, by definition.

From my point of view Glassfish ESB is not an opensource product with optional commercial support, it is a commercial product with the additional benefit of being opensource. Putting words in the right order makes a lot of difference sometimes.

Wednesday, January 28, 2009

The Growing of REST

Here I try to give my point of view on why REST is gaining a lot of traction against SOAP in the Webservices world.

First, I have to explain that REST is an architectural style while SOAP-based WS are just a way to implement RPC, which is the most basic inter-communication style. Even Corba one day went to the point of being very mature an stable, but how many new projects are now started with it? A technology should demonstrate itself within a reasonable time window, otherwise people get tired. My first SOAP implementation, I think it was 2000, was with Axis's ancestor Apache SOAP: I still feel the headache.

My opinion is that most self-proclaimed RESTful implementations are not really so, because REST is not just XML over HTTP (it can use JSON, plain text, Atom, binary streams, or a combination of all) but it is a way to implement an Object-Oriented architectural approach on top of Web resources (the data is the resource itself, the methods are the HTTP operations get, put, post delete, etc...). SOAP WS instead, as said, is plain, old RPC and it does not mandate to use HTTP only, but also works with other protocols such as JMS, etc.

Now I hope we agree programming languages evolved from procedural to OO, so I'd say the REST paradigm is more modern than RPC one, but we know that good OO is actually harder than procedural (and I mean good OO). So, IMHO, the traction REST is gaining is not due to the fact that it is conceptually simpler (as OO is not, if compared to procedural programming) but instead to the fact that REST is based on a simpler implementation, robust protocol (HTTP), few consistent operations (HTTP methods), a proven architecture (The World Wide Web, not a tiny one really...), a set of extremely tested technologies (Web servers, Web proxy and caches) and a long track of true interoperability and security (again, the WWW and SSL). Honestly I am very happy that I still access my on-line banking through HTTP + SSL and not any WS-Security......

Resource Oriented Architecture, implemented through REST, is actually conceptually harder than RPC style for the same reasons that proper Object Oriented modelling is conceptually harder than plain procedural programming. But a lot more rewarding, if we compare the final outcomes. And that's why, by the way, also good stuff like Spring is gaining a lot of traction: it allows to model a real domain through POJOs.

Maybe I am too old, because in this procedural world of SOAP and EJB (yes, EJB specs before version 3 forced a procedural programming style) I still persist thinking that Object Orientation should be the way to model business domains properly: I learned that many years ago and still nothing has been able to change my mind. Of course, integration often demands for a simpler paradigm than OO: probably scripting will happily rule here (IFL and Fuji anyone?).

My opinion is that the WSDL + WS-* combination, regardless of all the efforts, is still a way too complex, real interoperability is still questionable and security is largely unproven, if compared with plain, old HTTPS and its some billions of daily Web transactions. However, I agree things are improving, but while I'm at customer site I'd love not to spend days resolving WSDL and WS-* quirks and instead focus my attention to more added-value activities....

More simplicity quite usually equates to more security.

Sunday, January 25, 2009

Tom Barrett's Open ESB and Mural Tutorials

Sun's Tom Barrett has produced a set of very well written and easy to follow tutorials, which can gently lead the reader toward a step by step introduction with many of the newest OpenESB related technologies, together with Netbeans and Glassfish. Good job Tom!

Tom Barrett's OpenESB Tutorials

Friday, January 23, 2009

Dump Glassfish HTTP conversations to debug security

In Glassfish is possible to dump the HTTP conversation to debug Web Services, by adding the following two JVM options to domain.xml:

<jvm-options>-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true</jvm-options>
<jvm-options>-Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dump=true</jvm-options>

Then when the web service is called the server.log will show complete SOAP envelopes for both request and reply:

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
---[HTTP request]---|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
Host: localhost:8080|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
Content-type: text/xml;charset=UTF-8|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
Content-length: 866|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
User-agent: Jakarta Commons-HttpClient/3.1|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
Soapaction: ""|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:svc="http://xml.netbeans.org/schema/svcMessages">
<soapenv:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><wsse:UsernameToken wsu:Id="UsernameToken-27540698" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsse:Username>wsitUser</wsse:Username><wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">xhM3irVV+4ROPhzEjwlEeL7IViI=</wsse:Password><wsse:Nonce>oE1wwvm+ZLeBuvgDJvaLvw==</wsse:Nonce><wsu:Created>2009-01-21T02:13:32.238Z</wsu:Created></wsse:UsernameToken></wsse:Security>
</soapenv:Header>
<soapenv:Body>
<svc:payload>Maurizio</svc:payload>
</soapenv:Body>
</soapenv:Envelope&gt;
|#]

[#|2009-01-21T03:13:33.783+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|--------------------|#]

[#|2009-01-21T03:13:34.828+0100|SEVERE|sun-appserver9.1|com.sun.xml.wss.logging.impl.filter|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;_RequestID=9ffc6c88-9210-451f-ad64-08ec2bc5ca74;|WSS1408: UsernameToken Authentication Failed|#]

[#|2009-01-21T03:13:35.015+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
---[HTTP response 500]---|#]

[#|2009-01-21T03:13:35.015+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=25;_ThreadName=httpSSLWorkerThread-8080-0;|
<?xml version="1.0" ?><S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><S:Fault xmlns:ns3="http://www.w3.org/2003/05/soap-envelope"><faultcode xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">wsse:FailedAuthentication</faultcode><faultstring>Authentication of Username Password Token Failed</faultstring></S:Fault></S:Body></S:Envelope>|#]

Monday, January 19, 2009

Alistair's top ten ways to know you are not doing agile

I stumbled upon the following not-so-new blog entry by Alistair Cockburn, a nice short list of "bad smells" for lack of agile project management. These days companies are still frequently trying to engage project managers asking for arcane list of certifications which are mostly theoretical and paperwork-oriented. Alistair, who has written one of the best book ever about Use-Cases, instead insists on agile methodologies and expert leadership as the only effective way to deliver projects.

In my experience pragmatism, agility and the ability to deeply dig for real business requirements are the most important factors for successful integration projects. For effective consulting is mandatory to keep open a constant, daily communication link with business experts and project stakeholders, relationships with just customer's technical people almost never provide for the necessary unbiased information.

Too many developers are just trying to experiment with the latest "cool" technologies, only because they want to fill a row in their CV, regardless of the real business needs of their employers. In my experience these days two of the biggest CV-fillers are SOAP Webservices and BPEL, as I rarely see those technologies used properly.
Not that they couldn't be useful, but:

"When all you have is a hammer, everything looks like a nail."

SOAP WS and BPEL are the nowadays' hammers.

Monday, January 12, 2009

Buzzwords demystification

Words like "SOA", "ESB", "EDA" are common and, unfortunately, extremely abused. In summary, below a one page description of the main buzzwords floating around.

Architectural Styles

SOA is an architectural style aimed to create a conceptual framework and a collection of practices to improve the alignment of IT to core business needs. SOA is not about technologies. At its core there is the concept of Service, which is an abstract interface to a concrete business function, for example: logistics management, HR & recruiting, stock control, order management, invoicing, ... A Service implementation is governed by a policy, exposes an end-point, implements a defined contract and sends / receives messages. A service is transactionally atomic by definition, otherwise it's a process, which actually is about atomic services orchestration, so a very different and higher-level concept than service itself.

ESB is an architectural style aimed to highly decouple clients form service implementations, by creating an uniquely identifiable space where service policies can be applied uniformly and managed consistently.

SOI Service Oriented Integration is an evolution of classical EAI concepts, introducing an higher level of decoupling among applications by connecting abstract interfaces defined by service contracts instead of mere point to point connectivity usually happening within EAI traditional solutions. SOI = EAI + SOA.

EDA Event Driven Architecture is an architectural style which is the evolution of previous MOM techniques toward SOA concepts, by using one-way asynchronous message exchange patterns. It is a way to model the typical real-time, asynchronous system interactions happening in the real world. Routing, filtering, pipe-lining of messages are the typical tasks of an EDA, activities like CEP (Complex Event Processing) are sometimes necessary.

In Conclusions

Architectural styles and practices are the "what", "who", and "why" of a solution, they shouldn't be confused with technologies, which are the mere "how". Still we read around or listen to people who think that to make his/her IT infrastructure better and more fashionable they need to implement a bunch of SOAP Web-Services + some BPEL orchestration... That is deadly wrong and has driven the acronym SOA to be abused and inflationed and a lot of projects to fail.

Updates: yesterday Paul forwarded me this very interesting article about how Bechtel Corporation is implementing a huge SOA effort to create an internal SaaS infratructure.

Wednesday, January 7, 2009

The word “SOA” is dead

As usual Anne Thomas Manes wrote a brilliant and controversial article about SOA:

"Although the word “SOA” is dead, the requirement for service-oriented architecture is stronger than ever.

But perhaps that’s the challenge: The acronym got in the way. People forgot what SOA stands for. They were too wrapped up in silly technology debates (e.g., “what’s the best ESB?” or “WS-* vs. REST”), and they missed the important stuff: architecture and services."

I humbly wrote something similar before, I was discussing about governance needs but the problems are almost the same:

"IT is broken: we clearly see this simple and true fact daily, but I hardly find organizations able to recognize the fact that they need to reconcile their IT operations with their business needs first through organizational changes. They too often just try to buy the next magical box and hope that plugging it into their IT department it will fix problems.

Again and as usual: lot of money, few successes and SOA is becoming another abused bad word."

However the concrete requirement, as Anne wrote, is still here and unresolved.

Updates

Anne's article is creating a lot of discussions on blogs, here time by time I will collect some of the most interesting reactions.

Steve Jones's In a recession its even more about the services:

"Reading between the lines of what Anne writes I think I'd say that her statement is that vendors are moving away from SOA as they've flogged you enough stuff and now they want to flog you its offspring: mashups, BPM, SaaS, Cloud Computing
[...]
the marketing fury of T-SOA has moved on as their just aren't that many more ESBs and Web Service tools that you can be sold.
[...]

If you adopt the new technologies without having a services mentality then you will create a degree of mess that will make the one that consultants and vendors got fat on with EAI look like a trivial problem. Doing Spaghetti inside your firewall in big applications is one thing, doing it over the internet and with thousands of small ones is a completely different scale of problem.

So in a recession you need to Identify your services, understand the business value that they deliver, understand the cost model to deliver that value and then decide on the right technology approach.

If that isn't SOA then I don't know what is. So in reality its the "other" SOA that is dead, not the SOA of today."

Yes, that's the point: vendors are looking for the next cash cow. But if you don't want to be treat like a cow you shouldn't behave like a cow and stop eating all the hypes vendors are always throwing around. SOA is actually a good thing, but it is not something new and does not require special tools unless you have all your requirements clear. Is really SOA that different from old object-oriented domain modelling? Isn't that about clean separation of public interfaces from their implementations?

A deep and savy analysis is presented by Gary Barnett:

Essentially - this is what I have to say -

  • Saying “SOA is Dead” is as meaningful as saying “Art is dead”
  • It is either misleading or foolish (or both) to state that SOA is dead
  • Giving up on architecture just because it’s a “chore” is the road to failure…
  • If SOA is dead, then Web 2.0 is dead
  • The good news, though, is that SOA is as fit as a fiddle
  • The bad news is that if you want to succeed, you have to be willing to make an effort
  • We have a simple choice: get it right, or do it all over again
Gary is not really distant from what Anne wrote, to me he just thinks her message can be misunderstood. I personally already realized in 2007 that the acronym "SOA" was seen with growing suspicion by customers, so I was suggesting my colleagues to start discussing more concrete aspects. As Gary wrote, in enterprises the need for Architecture is stronger than ever, it is not time to give up but maybe to just re-tune our vocabulary.

Gary writes:

"Architecture is pivotal to the success of service based IT. It’s not a “nice to have”, it’s a fundamental necessity. Unless we as an industry make a real commitment to architecture we’re doomed to continue to make the same mistakes we’ve been making for decades."
[...]
"Take mash-ups, as an example – Sure, you can quickly assemble some fantastic apps by simply grabbing some javascript and coding away. But there are big, big, disappointments in store for mash-up builders who assume that because it’s cool and new they don’t need to think about all those boring questions that a mature approach to architecture helps you to answer (Reliability, security, availability, scalability, re-use etc etc)."
[...]
"As growing numbers of people succeed, we’re learning (or re-learning) a host of lessons about how to deliver the agility, re-use, scalability and general goodness that a service oriented approach to software architecture, design and development can provide."

Read his complete blog article.

Friday, January 2, 2009

Integration Projects Usual Madness

Joel Spolsky wrote a classical article about writing better code. The 12 Joel's rules are below summarized:
  1. Do you use source control?
  2. Can you make a build in one step?
  3. Do you make daily builds?
  4. Do you have a bug database?
  5. Do you fix bugs before writing new code?
  6. Do you have an up-to-date schedule?
  7. Do you have a spec?
  8. Do programmers have quiet working conditions?
  9. Do you use the best tools money can buy?
  10. Do you have testers?
  11. Do new candidates write code during their interview?
  12. Do you do hallway usability testing?
EAI projects pose additional serious challenges in terms of project management and quality assurance. Even if the term "EAI" seems to be out of fashion nowadays, I think the majority of projects which have been labelled as "SOA efforts" are, in facts, still plain, old EAI.

EAI Quality Assurance and Testing

Especially when doing EAI, unit testing is not enough. Joel is in favour of developer's unit testing and personally I think using a tool such as JUnit can prevent a huge numbers of bugs, but unit-testing integration flows is often much harder than unit-testing regular applications. This is due to the fact that when implementing EAI it is necessary to deal with a multitude of external systems which are out of our domain of control. Mocking those systems is often very hard or even impossible, it could lead to write and maintain more mocks than actually manageable and, additionally, the quality and realism of those mocks could be at least questionable.

Testing activities for EAI teams is a matter of organization and operations, because often it could be a cross-department issue. For example, testing SAP BAPI interfaces requires some commitment and lot of assistance from the SAP team, which in the meanwhile could have different priorities. The same for other legacy technologies. This clearly scales to a management issue, the EAI team has to be allowed to orchestrate the testing and validation process with all the involved external parties. Cross-department activities are usually difficult for organizational reasons, not much for technical ones.

Critical Success Factors


While at SeeBeyond we had some specifically integration-related rules, which were useful to establish what we called "Critical Success Factors":
  • The executive sponsorship must be established up front to facilitate timely decision making, appropriate resource allocation and removal of business barriers.
  • Establish an integration team with a balanced mix of IT, end business, management and consultants who will work together toward the main goal of a successful implementation
  • Provide just-in-time training and coaching to the appropriate people on operation of the product components, keeping them ahead of the learning curve,
  • Field level buy-in to the implementation is critical to the success of the project
  • Executive management must communicate their commitment to the new technology and illustrate this commitment through their actions.
  • Availability of key stakeholders to facilitate the decision making process and to keep the implementation moving forward.
  • A complete test and validation plan should prepared in advance. It is not up to the single developer to validate his own work, QA is an overall function.
  • As this is an integration project, all interfaces must be defined before starting the implementation.
  • Changes have to be accommodated into a change management process which takes care of the deadline. All the interfaces should be unit-testable and end-to-end testable as being part of a higher level process.
  • Implementation will proceed by very short iterations of analysis, design, development, test and deployment. At the end of each iteration a working prototype should be ready to run functional validation tests.
Regarding SOA, reports say that most SOA initiatives are failing because of inadequate business involvement and commitment. EAI is a better understood discipline (really?), but still projects tend to fail for exactly the same reason as newer SOA efforts.

Missing Deployment Plan

Just after the functional specs and scope, it is important to define a Deployment Plan and a Deployment Architecture:

"The deployment project plan establishes a baseline reference plan. It is your road map for the deployment project. The roles and responsibilities of each organization, the schedule of tasks, and any estimates must be defined in this plan.
[...]
Deployment Architecture (DA), together with Requirements Analysis, is the most important component of your plan, and should be documented carefully. Deployment begins when you plan and schedule how, in view of your analysis and allocated resources, you want to implement your environment. During this phase, you set up the operation procedure and schedule for the entire deployment project. DA provides guidance to the successive design and implementation phase, so DA creation always must precede it. In fact, developers needs to use the DA document as a guide for implementing their flows, adapting services communication to the DA. Of course, sometime it is necessary to adjust the DA to accommodate design needs emerging after some development, but if this happens frequently then it is a sign of a bad original DA, so it is time for a deep review.
Poor analysis and planning can cause major problems during the later phases. Thorough, comprehensive analysis and planning can make the deployment project easier, more efficient, and less costly.
"

The above is a mix of statements from the Java CAPS documentation plus some comments of mine.

Organizational Issues


In my experience most of the issue I have encountered during Integration projects are organizational and operational:
  • Lack of organizational information
  • No complete specifications
  • Not enough back-end information to fill the specifications
  • Lack of coordination
  • Frequent changes in project organization (document store moving, too many documents everywhere, anonymous doc.)
  • Huge integrations are thrown to single developers
  • Lack of commitment
  • Sometime is difficult to understand who is in charge
  • Developers are frustrated by changing deadlines
  • Bad or totally missing test data!!!
Missing Domain Model

You wouldn't write SQL queries before modelling any database relationship, would you?
An usual requirement for Services is to build first a Domain Model as a common, business oriented standard communication device. This is very close to good Object-Oriented programming practices, but Service-Oriented is focused on exchanging documents instead of objects. A domain model can be designed by normal entity-relationship diagrams and, for example, implemented through XML-Schemas.

Common Misunderstandings

  • Integration projects are like software development projects
>> No! they are very different!
>> Integration projects are cross-department projects
>> Need more involvement
>> Need more cross-management
  • Bugs in software are all the evil
>> No! Half of bugs are not real platform's bugs but behaviour due to bad design/assumption.

Integration project with any vendor will fail if there are no clear specifications...

Inter-team communication madness

  • Every team has its own life
  • Not coordinated at all
  • No motivations to help
  • Mostly Anonymous functional and technical documents !?!
Finally... Some Recommendations
  • Long term strategy but short delivery cycles
  • Top-down, business-driven analysis
  • Short iterations, deliver value within few weeks
  • Break big projects into smaller pieces
  • Stick to deadlines! Avoid changing a deadline!
  • Instead, what features can fit into the deadline?
  • Adopt a Project Management methodology!
Some of the above statements could seem controversial. What about deadlines? there is nothing more frustrating for a developer than working 80 hours per week, weekends also, just to discover that a couple of days before deployment some manager decides that the feature is not really that important and can be delayed. Boy, you are not going to catch me anymore! So my advice is: fix the deadline and instead move in and out features, keep the pace, sustain the mood.

Always ask ourself why? "Why?" is extremely powerful: before start fighting to solve a technical issue one should ask why I'm doing this? Is there any better, cleaner design which could alleviate or even cancel the technical issue? Do I really have all the necessary information? Am I answering a question nobody asked for?

Conclusions

Integration projects are somehow quite different from application development projects:
  • Interfaces and meta-data must be clearly defined
  • Service contracts must be defined (SLAs)
  • Test data must be readily available!
  • External systems experts must be available!
  • An architectural board in charge of taking decisions needs to be established.
It seems that the above list is exactly what we need in case of any project, application development or integration. But badly-managed dependencies with the external world can sink any integration project regardless of how good is the team and its internal process. Integration is about connecting existing interfaces: what can I do if interfaces are fuzzy and undefined? Frustration, that is....

Thanks goes to my friend Juraj Kazda of Sun, for discussing and writing down these issues while working together in several projects.