Distributed SOA

Nice article on distributed SOA

It's hard to turn anywhere today without seeing something about
service-oriented architecture, or SOA, and a debate about the "right"
way to do it. That's not surprising, I guess, since every new
computing trend generates debate and every vendor tries to promote
their technology as the right technology – the right product – that
will allow customers to take the most advantage of a new approach.
It's a scramble as vendors try to aggressively reposition their
existing product suites to take advantage of customer interest in a
hot IT trend. It's unfortunate, though, because this behavior often
creates a lot of confusion and potential disillusionment, as promises
made often do not turn into promises kept and technology solutions
sold as appropriate for SOA may turn out not to be.

To set the right perspective on this, it's important to note that SOA
is, by definition, distributed. The purpose of a service is to
communicate remotely with another service, typically to share data.
The purpose of an SOA is to change the approach of IT from building
bespoke, monolithic applications to building applications that are
developed and integrated more and more using assets from a collection
of shared, reusable functionality, i.e. services.

A distributed SOA infrastructure represents the easiest way to deploy
and consume shared, reusable services, facilitating incremental
adoption (both economically and technically) and enable greater
deployment flexibility, adaptability and maintainability (e.g., single
services can be certified for update more easily than an entire
application) .

Unfortunately, centralized approaches to SOA infrastructure continue
to be developed and proposed. Vendors will go to great lengths to
convince the technology buying public that what they're offering is
already SOA compliant, always was, always will be and was truly
designed from the beginning to facilitate their customers' move to
SOA, no matter whether it was originally designed to be a JEE
application server or EAI system.

In other words, vendors taking an opposing view to distributed SOA
infrastructure often do so because that's the nature of the software
infrastructure they already have. A refried EAI hub or JEE-based stack
or any other solution that requires request messages to be passed
through a central control point, cannot be considered truly
distributed since all access to services must be routed through a
central hub or a server. This centralized approach to SOA adds cost,
limits reuse, reduces flexibility and introduces a potentially
expensive bottleneck. In the worst case it could even negate the
reasons for moving to SOA in the first place. People are bound to be
disappointed if the flexibility of their SOA infrastructure does not
meet their requirements.

One only has to look to the World Wide Web to see an example of a
distributed application that very successfully meets its users'
requirements. The Web is the largest distributed application ever
built and it's distributed by nature, like an SOA should be. When you
use a browser to click on a link to a URL, your request is not routed
through some kind of central controlling authority deployed in a
server or a hub. Your request goes directly from your browser to the
Web server hosting the page you have requested. This approach works
fine for the Web and it also works fine for an enterprise's SOA. Web
endpoints can be updated individually without breaking clients,
affecting other sites or requiring any update to a central hub or
server, since requests do not have to pass through one in the first
place. A good SOA infrastructure also supports these capabilities.

Fortunately, an infrastructure solution exists that embraces the
distributed nature of SOA. The distributed approach to SOA
infrastructure uses smart endpoints that service enable applications
and allow them to find and directly communicate with other services.
Enterprise qualities of service, such as high availability or advanced
security, can also be included in the endpoints to preserve the
capabilities upon which existing mission critical applications rely.
Distributed SOA infrastructure is about creating an IT environment
that is platform neutral, standards-based and highly flexible, so that
it can respond better to an ever-changing technology and business
landscape. A distributed SOA environment is therefore better able to
support the technical and economic requirements of an SOA based
application. Finally, a distributed approach to SOA infrastructure
allows you to move at your own pace, deploying services one or two at
a time, adding features such as orchestration, registry/repository ,
management and so on as needed and not before.

I'm not saying that EAI systems, hub and spoke based applications or
JEE server centric approaches to SOA infrastructure are all bad or
wrong. Many times existing corporate application functionality depends
upon one or more of these implementation styles. All I'm saying is
that a good SOA infrastructure shouldn't constrain your design options
to what any one of these things can do – in fact a good SOA
infrastructure embraces and includes them among the collection of
reusable service assets.

A parallel between the benefits of distributed SOA and the airline
industry can be seen in the way in which the low-cost operators are
challenging the established carriers. The approach of the established
operators is based on an expensive hub and spoke model that funnels
passengers through a small number of dedicated travel hubs. Large
planes that cost more to operate, fly from spoke airport to hub
airport, where passengers are processed for further travel to their
final destinations. In this model the planes cost more to operate and
airport facility charges are higher. With the rise of low-cost
airlines, which operate on a more distributed, point-to-point model
(smaller planes directly flying to smaller airports), the airlines
that are tied to the traditional hub model are taking a significant
financial hit.

SOA customers do not need more of the same old bloated and expensive
software stacks. Customers need better software specifically designed
for the requirements represented by the SOA trend – to improve the way
in which existing (and new) IT functional assets are provide to
applications. SOA designs need a good way to create and deploy
reusable services that can be called upon simply and directly when and
where needed. Customers need low cost options that allow them to start
small with incremental adoption, using point-to-point communication
solutions that avoid expensive new servers and hubs, adding quality of
service and other features as required. In short, they need SOA
infrastructure that really meets the inherently distributed nature of
an SOA.

You can read the above article here

Socio-Political and Commercial aspect of WS/SOA

Nice article and a good retrospective.

http://atownley.org/2006/12/socio-political-and-commercial-motivations-for-ws/

Things to know about SOAP before using it - Very funny though

Here is the most admired post, put together nicely about what SOAP is all about !! Don't hate me if you are a SOAP proponent.

RMI, CORBA and DCOM

Gregg Wonderly brings out a clear distinction among RMI, CORBA and DCOM usage in one of the tech discussion forums:

I really find it interesting that people continue to bash RMI as something to
put in the same space as CORBA and DCOM. Clearly people have used RMI in much
different/worse systems than I. Based on some of the arguments we've had here,
it seems that the biggest problems people have had with RMI are based on
misunderstandings of the technology and hence misuse of the features. A lot of
people seem to have just made every object extends UnicastRemoteObject, or used
that class to export everything, instead of looking closer and granularity. The
other falling off point is serialVersionUID and versioning overall.

One of the key things about the RMI programming model is that version #1, had
many limitations. Version #2 is present in the Jini platform, created by the
engineers who were part of the CORBA system design and development as well as
RMI. So, they know all of the issues that people waved in their faces over the
years.

The predominate arguments against RMI are always based on "granularity". With
RMI programming, you can design systems that pass big objects, no objects, small
objects, lots of text, or whatever you want. What Jini gives you in an RMI
programming model, is the ability to (with published specs and standards), plug
in any endpoint or invocation technology that you need. Thus, you can take
advantage of all the features of the Jini platform while still providing outward
interfaces using other technologies such as the web or WS-*.

CORBA is a invocation layer technology. DCOM is an operating system specific
invocation layer technology. RMI might be considered an invocation layer
technology. But, it is actually a programming model which includes something
much larger and that's mobile code. Jini expands the RMI programming model with
more features and provides services that use the underlying technologies to
really expand the possibilities.

Lumping RMI in with CORBA and DCOM, I think, really hides the power of what is
available in that programming model. In particular versioning is greatly easier
to deal with when you can exploit mobile code to allow clients to continue to
see an old version of the interface while the implementation is actually much
different. Providing this level of flexibility is what makes it possible to
grow a service with more features and still maintain backward compatibility.

Google-Maps is a great example of the use of mobile code for version control. I
have old google-maps which use old versions of their interface. I haven't had
to convert those to the new system because I download mobile javascript code
that manages that issue for me.

At some point, they might retire that support and I'll have to switch. But, the
point is that there's not an instantaneous moment where I have to be compatible
just so that they can move on to create a better system. They can do it
continuously.

Duties of chief Software Architect

My recent post on duites of Chief software architect is availble here

Developer Or Architect ?

I stumbled across this nice article trying to define developer and architect !!

Microsoft launches Architect Certification Program

Being a Sun Certified Enterprise Architect, I am aware of very few certifications available in the market. I went through the criteria needed to get Microsoft Certified Architect(MCA) program and looks interesting.

Web Development Directory structure

We have started the new web development and I was discussing about the directory structures with the leads. Initially we had created the directory structure based on technology. I would like to call that as "Technology based directory structure". Recently while we were reviewing one of the open source product, we found that there directory structure is based on "Requirements/Features". Combining the above two we came out with Requirements-->Technology directory structure.

Examples:
Technology based directory structure:
Service
|___ api
|___ impl
|___ dao

Feature driven Directory structure:
Service
|___ usermanagement
| |_ api
| |_ impl
| |_ dao
|___ groupmanagement
|_ api
|_ impl
|_ dao

Java Heap and Windows Operating System

Today we were trying to deploy confluence integrated with Valtech cockpit on production. The result was an eye opener. Thanks to the confluence which consumes enough memory to push the heapsize to the limit.

As we were doing a test integration, we encountered our first "Out of memory" error within the first 15 minutes. I went back and tweaked the cache parameters within confluence by reducing some of the params like "time to live in memory" and also "number of objects to be in memory". I felts it worked great for next 24 minutes. Even though the "System information" section showed the "heap size" being used only 83%. Suddenly it crashed by throwing "Out of memory" exception.

This tweaking continued until we were able to make it work for next 45 minutes and, we felt this does not make sense to fight without proper information in place about the caches. Anyways, the point is during this struggle to make it work I found this article about setting memory size and its relation with the operating system. I know now, for serious application deployment we should think of linux/solaris or 64 bit systems.

Impact of rewriting the code from scratch

 

  I was going through Joel's
website, and came across the cardinal law  of programming.

        It's
harder to read code than to write it



In fact this feels so correct. In the morning, I was trying out ways to
build a javascript function for "inline editing" in one of the custom
grid we have build. I found a sample source code on the website, and when I
opened it, the first thing I felt was this looks like a mess, there are no
proper comments, code is scattered !!!  I prefer to just understand some
part of the code and rewrite the whole stuff by myself rather than reusing the
same, which holds good for many developers.



As human beings each one of us are unique, and I believe the same applies for
us when we take the roles of programmers. We want to be unique.



Result: 1. A lot of time gets wasted in rewriting the code

2. Disaster can happen like for Netscape, where it seems they tried rewriting
Netscape 4.0 and it was all over the place, and it took nearly 3 years for them
to put there code together to release Netscape 6.0 (Jumping 5.0), and now they
have lost nearly 80% of the market.



Moral of the story: Never plan to rewrite an entire product from scratch
without clearly analyzing the impact

Testing the class that uses API and not the API itself

I read the following post from Uncle bob's Website:


It's not enough to write tests for an API you develop, you have to write unit tests for code that uses your API.

When you do, you learn first-hand the hurdles that your users will have to overcome when they try to test their code independently.

Bulletproofing SOA

I found this nice article on the web on how to bulletproof SOA:

http://news.zdnet.com/2100-9593_22-6076996.html?tag=zdnn.alert

SOA 2.0 - Moving towards Event driven architecture

Here is an article about future of SOA 1 i.e. SOA 2.0

"SOA 2.0 is the term that we're using to talk about the combination
of service-oriented architecture and event-driven architecture," said
Steve Harris, vice president of Oracle Fusion middleware.

The term, SOA 2.0, also is being championed by Gartner's Yefim Natis,
a vice president and distinguished analyst at the firm. Contacted by
telephone, Natis stressed event-driven architecture as the main
distinction between SOA 2.0 and the first, client-server driven
iteration of SOA.

"SOA as we know it today deals with a client-server relationship
between software modules," with services being subroutines serving
clients, Natis said. "However, not all business processes and
software
topologies fit this model."

With SOA 2.0, an event-driven architecture is deployed in which
software modules are related to business components, and alerts and
event notifications are featured. The initial SOA concept has not
been
event-driven but instead has featured direct calls from one piece of
software to another in a client-server process, Natis said. SOA
implementations have focused on Web services and subordinates to
clients, he said.

SOA 2.0 applications could include order processing systems, hospital
admissions processes or bank transactions, Natis said.

Oracle is positioning its Fusion middleware components as a solution
for SOA. Oracle sees the Java Platform, Enterprise Edition 5 (Java EE
5), SOA 2.0 and Web 2.0 coming together to produce a more productive
application platform, said Thomas Kurian, Oracle senior vice
president. Web 2.0 features more dynamic clients.

You can read this in full at:

Cruise Control Step by Step procedure

Following steps might be useful while installing cruise control:

Please Note:
1. These steps were created in one of our projects and by a team member. Many of the folders are hard coded. The developer needs to modify the folder names appropriately.
2. There could be other ways to set up and more easily.

Installation of the Cruise Control
-----------------------------------
Version : Cruise Control 2.3

INSTALL PROCEDURE

NOTE: It is VERY, VERY important that you follow this procedure carefully and exactly with NO changes.
CruiseControl can be quite finicky and takes a long time to debug if it doesn't work right!


Pre-requisite: (on the machine where cruise control is running)
---------------------------------------------------

Tortoise CVS should be installed.
Tomcat 5.0 is installed
JSDK1.4.2 version or above
ANT 1.6 or above


Folder Structure
---------------------------------------------------
-Create a Folder continuousintegration on c:
c:\continuousintegration
-cruisecontrol
-builds
-artifacts
-checkout


Installation Steps:
-------------------

1) Create the Folder structure as shown in above.

2) Install the Tortise clint on the machine where the cruise control is running.
(required to update the files from repository)

3) Install the Java SDK 1.4.2_06 in "c:\j2sdk1.4.2_06" and set JAVA_HOME to the same.
or set the JAVA_HOME where you have the JSDK installed.

4) Install Tomcat 5.0 in "c:\Tomcat5.0" (no spaces!) and set CATALINA_HOME to "c:\Tomcat5.0". Use port 80 during the installation and note down the administrator username and password tht you supply(username:admin password: admin).

5) set ANT_HOME variable in the System's environment to "C:\continuousintegration\apache-ant-1.6.2"
or set the ANT_HOME where you have the ant folder.

6) download the cruisecontrol-2.3.0.1.zip version from the below link and extract this folder in the path
C:\continuousintegration\cruisecontrol downloadlink :http://cruisecontrol.sourceforge.net/download.html

- And set CCDIR variable in the System's environment to
"C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main"
(This is path of the cruise control which you extracted)

- copy the config.xml file attached with this document into path
"C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin"

7) Prepend "%JAVA_HOME%\bin;%ANT_HOME%\bin" to the PATH variable in the system environment

8) Restart the machine. This is necessary for the PATH changes to take effect for creating a Service


9) Start Tomcat. Go to "http://localhost/manager/html" and login in using the administrator login and password.

10) Before deploying the cruisecontrol.war on tomcat, build the cruisecontrol.war through command prompt.
follow steps
-go to command pomopt "C:\Downloads\cruisecontrol-2.3.0.1\reporting\jsp"
-execute command C:\Downloads\cruisecontrol-2.3.0.1\reporting\jsp>build war;
-build will start now and will ask for the fallowing inputs so plese input associated values
as given below.
-set.log.dir:
"C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin\log"
-set.status.file:
"currentbuildstatus.txt"
-set.artifaacts.dir:
"C:\continuousintegration\cruisecontrol\builds\artifacts"

-confirm that your build is successfull.

11) Using the "WAR file to deploy" section, deploy
"C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\dist\cruisecontrol.war" into
"C:\Tomcat5.0\wepapps" Tomcat. You should be able to see the cruisecontrol jsp application in a browser at "http://localhost/cruisecontrol". This does NOT mean that cruisecontrol process is running.

12) Check out your project, and copy/cut the entire "checked out source code" folder and paste it into "C:\continuousintegration\cruisecontrol\builds\checkout".

13) Go into the "C:\continuousintegration\cruisecontrol\builds\checkout\" folder and run "ant war" to verify that an ant build works in this directory.

14) Run "InstallCruiseControlService.bat" from directory "C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin" in a command shell.

15) If you go to "Start->Control Panel->Administrative Tools->Services", you should now be able to see a service called "CruiseControl".

16) Change the service to "Automatic" and start it. This should run the service. Check by typing "http://localhost:8000" into a browser. This should bring up the cruisecontrol's JMX control panel.

17) Click on the link named "CruiseControl Project:name=" To view the attributes set for the cruisecontrol.

18) Run "cruisecontrol.bat" from directory
"C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin" in a command shell.
If the console did't throw any exception then Hopefully your cruise control will be working fine
and it will continously look for the modification at your repository.

19) See your console for your cruise controle activities or you can see the log files from
C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin\logs

Some of the Usefull Reference Sites:
http://cruisecontrol.sourceforge.net/
http://cruisecontrol.sourceforge.net/main/configxml.html
http://www.javaranch.com/journal/200409/DrivingOnCruiseControl_Part1.html
http://www.javaranch.com/journal/200410/DrivingOnCruiseControl_Part2.html

Configuration files
---------------------
1. config.xml ==> Generally available in: C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin

2. CruiseBuild.xml ==> available in your source code folder. Should contain the CVSRoot info.

3. use "\" appropriately while mentioning folder structure based on your operating system.

4. Take special care while specifying the environment variables. I got stuck while appending "%ANT_HOME%"\bin to the PATH. Finally, I expanded ANT_HOME to proper windows directory and pasted onto PATH.

5. Ensure that the log file directory is properly set. Mostly it would be:
"C:\continuousintegration\cruisecontrol\cruisecontrol-2.3.0.1\main\bin\logs\
You can also set the log directory at the time of building the Cruisecontrol WAR OR you can change the log dir in web.xml after deplopying Cruisecontrol on tomcat

Amazon being a technology company

All these days when somebody talked about Amazon, first thing I used to visualize was some software book image on the web browser !!! But following article changed the way I look at amazon, as a Technology company !!

http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=388&page=3

I remember reading it somewhere that McDonald's is not really into making burgers but are into real estate. Now I can see what I read !!!

Difference between Transfer and Transport protocol

Here is a good definition of transfer and transport protocol found on SOA discussion forum.

They're two different protocols of the stack that play
entirely different, non-overlapping roles. Transport is for
moving bits around the network. Transfer is for exchanging
data between applications. Transfer uses transport.

In OSI-speak (since somebody brought it up) IIOP, SOAP (as
commonly used), etc.. are layer 5/6 protocols, while transfer
protocols are layer 7. That's why I've always got a chuckle
about those try to put SOAP over HTTP 8-)

Big Upfront Framework design (BUFD)

Recently I have been thinking is it really worth doing Big Upfront Framework Design(BUFD).
I have taken part in designing several frameworks in my past, but never thought(or didnt get a chance) the best phase to design frameworks. It is agreed by default that frameworks are designed upfront before product development begins. Many highly intelligent architects, designers sit together for days and nights without proper sleep and design meta data driven,XML based,database independent frameworks!!!. But if we carefully look into this, upfront framework design is not really agile, and secondly, ROI is also bad.


* Doing Upfront Framework Design(UFD), would force the designers and developers to design application or develop application to fit the framework, rather than steering the development effort in satisfying the customer's requirements.

* UFD, is not profitable to the customer, because frameworks are generally designed to be as generic as possible keeping future requirements in mind, and due to which, customer would be spending money on "future" requirements in the present!!!

In fact I was trying to think can UFD be applied to Project development scenario, where the project's duration is less than a year ???... Now I feel that it does not suit here too. The reason being, any framework design and development would take atleast 3-4 months, and I feel this effort is not worth spending for a shorter duration project !!

Ok, the best possible solution could be:

* Start thinking about framework from the beginning (dont stop yourself), but dont start the design until you pass the inception pase.

* Start laying the foundation from Elaboration phase.

* As the product development progress, design, tweak the framework iteratively.

* The Framework should evolve over a period of time, and at the same time ensure that is "usable" (need not be complete) before construction phase !!

* keep refactoring, evolving and be agile

Cleared SCEA Part 1 Exam

March 31st 2006: I cleared the Sun Certified Enterprise Architect exam. I was planning to take up this exam from some time, and in the last 3 days I really put some effort and passed the exam with good scores.

I have scored 100% in many of the following objectives, especially the ones that I am passionate about like Design patterns, EJB, Applicability of J2EE, Legacy Connectivity(even though I dont like this subject I know a bit), protocols, messaging.

Exam Objectives

Section 1: Concepts
* Draw UML Diagrams
* Interpret UML diagrams.
* State the effect of encapsulation, inheritance, and use of interfaces on architectural characteristics.

Section 2: Common Architectures
* Recognize the effect on each of the following characteristics of two tier, three tier and multi-tier architectures: scalability maintainability, reliability, availability, extensibility, performance, manageability, and security.
* Recognize the effect of each of the following characteristics on J2EE technology: scalability maintainability, reliability, availability, extensibility, performance, manageability, and security.
* Given an architecture described in terms of network layout, list benefits and potential weaknesses associated with it.

Section 3: Legacy Connectivity
* Distinguish appropriate from inappropriate techniques for providing access to a legacy system from Java code given an outline description of that legacy system

Section 4: Enterprise JavaBeans Technology
* List the required classes/interfaces that must be provided for an EJB technology.
* Distinguish stateful and stateless Session beans.
* Distinguish Session and Entity beans.
* Recognize appropriate uses for Entity, Stateful Session, and Stateless Session beans.
* State benefits and costs of Container Managed Persistence.
* State the transactional behavior in a given scenario for an enterprise bean method with a specified transactional deployment descriptor.
* Given a requirement specification detailing security and flexibility needs, identify architectures that would fulfill those requirements.
* Identify costs and benefits of using an intermediate data-access object between an entity bean and the data resource.

Section 5: Enterprise JavaBeans Container Model
* State the benefits of bean pooling in an EJB container.
* State the benefits of Passivation in an EJB container.
* State the benefit of monitoring of resources in an EJB container.
* Explain how the EJB container does lifecycle management and has the capability to increase scalability.

Section 6: Protocols
* Given a scenario description, distinguish appropriate from inappropriate protocols to implement that scenario.
* Identify a protocol, given a list of some of its features, where the protocol is one of the following: HTTP, HTTPS, IIOP, JRMP.
* Select from a list, common firewall features that might interfere with the normal operation of a given protocol.

Section 7: Applicability of J2EE Technology
* Select from a list those application aspects that are suited to implementation using J2EE.
* Select from a list those application aspects that are suited to implementation using EJB.
* Identify suitable J2EE technologies for the implementation of specified application aspects.

Section 8: Design Patterns
* From a list, select the most appropriate design pattern for a given scenario. Patterns will be limited to those documented in Gamma et al. and named using the names given in that book.
* State the benefits of using design patterns.
* State the name of a design pattern (for example, Gamma) given the UML diagram and/or a brief description of the pattern's functionality.
* Select from a list benefits of a specified design pattern (for example, Gamma).
* Identify the design pattern associated with a specified J2EE feature

Section 9: Messaging
* Identify scenarios that are appropriate to implementation using messaging, EJB, or both.
* List benefits of synchronous and asynchronous messaging.
* Select scenarios from a list that are appropriate to implementation using synchronous and asynchronous messaging.

Section 10: Internationalization
* State three aspects of any application that might need to be varied or customized in different deployment locales.
* Match the following features of the Java 2 platform with descriptions of their functionality, purpose or typical uses: Properties, Locale, ResourceBundle, Unicode, java.text package, InputStreamReader and OutputStreamWriter.

Section 11: Security
* Select from a list security restrictions that Java 2 environments normally impose on applets running in a browser.
* Given an architectural system specification, identify appropriate locations for implementation of specified security features, and select suitable technologies for implementation of those features.

Who is an Architect ?

I found this interesting definition as per James Mcgovern:

think of a good architect as someone who really understands the systems they oversee - not just the classes and the collaborations but the real 'soul' of the system. They're the sort of person who, given a particular change in requirements for their system, will come up with three different solutions but will instinctively know which is the right one to go for (and will then find all the technical justification in terms of changability, performance, robustness, etc.) because they know the system so well. They probably love the system a bit too.

Interesting definition of Enterprise Architecture

I found this interesting definition of EA:

Enterprise Architecture is an infrastructure and a set of Machines constructed in order to manage a chaotic, dynamic, unpredictable, complex, organic, prone to error, frustrating, Enterprise IT, which has to support an ever increasing, dynamic portfolio of products and services, through constant "ASAP, Now, Right-Away" modifications of business processes.