Poster Design #2 Workbook Seven Customer Databases – An Integration Problem You are working as a consultant for Integrated Trading, a financial services firm. They have merged several times, and they have brought you in to straighten out their customer database problem. They have too many customer databases, one for each of their services. Many of the customers overlap several services. They want you to design a system that will integrate the databases. Their problem scenario is the following: Last week, Mellon bank called in to trade an equity portfolio. After they provided all the information to make the trade, they said they also wanted to sell some convertible debt. They were transferred to the convertible desk – the salesperson asked them all over again for their customer information. They were really annoyed, and complained that not only do they need to explain themselves every time they are reconnected in the company, but that their bills, reports, and salespeople are all different across the five different services they buy from Integrated Trading. They threatened to bring their business elsewhere unless they are treated as a single customer. You have found out that the customer databases are just one part of the issue. Each database is controlled by a complex customer application that involves large amounts of custom code, ranging in size from 150,000 lines of Java to 1,000,000 lines of COBOL. You have been told to come with a solution that does not involve changing this legacy code. A previous consulting engagement has revealed the following. 40% of customers only use one service. But 30% use two services, and 30% use three or more services. From a strategic perspective, you would like to seamlessly offer customers additional services, and thus you can justify the expense of solving the integration problem not only because it will retain the larger customers, but also because it can help generate more revenue. There are more details on the customers who use more than one service. In most cases, they do not call in and use both services on the same call. They may call back later in the day, or on another day. Customer information does not change very much. Adding a new customer to the database has to be a relatively quick process, or you lose the customer. After the customer is added, usually there are less than 12 changes a year. The databases themselves all have different ways of representing a customer – some as a number, others as a set of characters. Some databases have hierarchies of 1
customers – for example, Mellon, and subcategories of equity and fixed income. Other databases are flat, listing each contact as separate customer –for example, Mellon Equity, Mellon Fixed Income. Now that you have a sense of the context of the problem play out the various situations as described earlier. Use the iterative 3 steps of synthesis to define the containing system (the larger system), then learn about the problems and challenges of the contained system and eventually make sense of why the current situation is contributing to what was described above. Although the focus of the first iteration of our posters is on the problem space, in order to learn about the problem space, you may find yourself moving from problem space to exploring solutions and then back to the problem space as you gain more insights in your attempt to find different ways to solve the problem. As you move from the problem space to the solution space, try different things using different integration mechanisms to address the problem at hand. A good way to portray these 7 databases is to look at this diagram as a start.
Us er Us er
Us er
App
DB
…
Us er
…
Us er
Us er
Us er Us er
Us er
Us er
… Us
…
Us er
Us er
er
App
App
App
DB
DB
DB
Us er Us er
… Us er
Us er
Us er
Us er
… Us er
App
App
App
DB
DB
DB
These are the 7 separate databases with their Associated applications and their associated set of users
Figure 1
2
…
It would also be helpful to use this multi-tiered architecture reference to better understand how applications and databases may be tightly vs loosely coupled in both legacy and more modern architectures. The constraint of not changing the legacy code could come from the applications that are single tiered, i.e. tightly coupled with no logical decoupling among presentation, application and the database.
Multi-tiered Architectures Performance, legacy technology, politics, privacy, regulator requirements
1-tier (tightly integrated)
Multi-tier Loose Coupling
2-tier (Modularization , Specialization)
Applicati on
Databas e
3-tier (Modularization, Specialization, flexibility/Adapta tion)
Presenta tion
Applicati on
Databas e
Presenta tion
Applicati on
Applicati on /
N-tier (Modularization, Specialization, Flexibility/Adaptat ion, Performance?, Integration Challenges??)
….
Applicati on /
Figure 2
3
Exercise: With the above as the case description and the focus of the integration problem, it is time to start the Ackoff 3-step process of identifying the containing and contained system, exploring the problem from the outside in. Remember in Poster design #1, designing of a Personal Information System, conceptually your sketch of the containing and contain system would look like this
Personal and You & PIS Problem S ace
Your social Network
Professional Life Step 1: Identify the Containing System (Goals, Objectives, Constraints, Boundary, Environment) Step 2: Explore the Containing System (Activities, Challenges, Problems, Outcomes) – Structure and Process of the containing system Step 3: Disaggregate to understand the roles of the Parts play to the larger system – its functions co-produced by the structure and process of the contained system
Draw a similar diagram to show the containing and the contained system for this design problem.
Step 2 in the 3-step process calls for you to explore the containing system, the business of Integrated Trading. From the case description you should be able to play
4
out the activities that may happen from the customers perspective, i.e. the customer requiring service from Integrated Trading. Exercise: Write out the scenario narratives of key activities that you can pick out from the case description as well as other activities that you think are relevant to be included. (Remember, you are modeling the key activities the customers engage in that are shown to be problematic or frustrated the customers). Write as many as you can here…
As you play out these scenarios, you will come across the complaints/problems that the customers are experiencing (This activity is your effort to start with bottom up inquiry and then arrive at some topdown understanding of what you might have observed). As you wear the “hat” of a customer, you can’t help but switching between assuming the role of the customer/user to the role of the designer, looking at the structure and process of the current setup of all 7 database areas. This is the beginning of moving between step 2 and step 3 of the Ackoff 3-step process. Switching between those 2 perspectives are important steps as a designer to learn the perspective of the users and then translate those problems/challenges that the users experienced to challenges that you as a designer need to address in order to then address the customers/users problems.
Exercise: From playing out the scenarios and learning from the customer/users perspective and the understanding of the current process and structure of the 7 database areas, write down or point out the problems/challenges that the customers are experiencing and WHY?
Also write down WHAT you think is/are the problems of the current setup of 7 databases, taking into considerations the constraints, evolution of the 7 database areas and the differences in how they serve their customers.
5
Exercise: Sketch/illustrate and or describe the nature of these 7 databases (derived from the case description) as they relate to HOW they contributed to the current situation (it is important to note and highlight necessary constraints both from the business perspective, i.e. the needs of Integrated Trading and its users as well as the constraint imposed by the structure of the current database systems)
Also, if you can try to answer the question of WHY this might have caused the situation that Integrated Trading is experiencing today would be good as it would help you design solutions that better address the underlying problem.
At this point, if you had followed the workbook, you would have the following: 1. Scenario narratives of customers doing business with Integrated Trading 2. Through bottom up above and topdown to learn about WHAT might be the underlying problem 3. In the step 3 of Ackoff 3-step process, you have also explored the current structure and process of HOW the 7 database areas are used to serve the customers and hence have some ideas as to the nature of the problem It would be useful at this point to refer to Figure 1 again, this time focus on the key constraint of this design assignment, i.e. your design solution should not force all 6
applications to change. It would help to look at the diagram (Figure 3) below to get a better understanding of this constraint.
Loosely Coupled System
Tightly Coupled System (Legacy)
APP DB
Putting Any integration Mechanisms Between App and DB will have a High probability Of forcing the App to change
APP
Many options Available to Introduce Integration mechanisms
DB Figure 3. As shown in figure 3, all tightly coupled system (one or more of the 7 databases have such constraint) are not designed in a modular fashion to recognize the interface between the applications and the databases. There are no interfaces or API or library to call on to work with the application or the database. Any of such intervention will inevitably make changes to the existing tightly coupled system thereby violating the constraint of not forcing all applications to be rewritten. However, if one or more of the 7 systems are loosely coupled (i.e. modular in design), you would have more integration options. As you begin to explore different options of integrating these 7 different databases which would in turn improve the business of Integrated Trading, you should also have read the “Integration Chapter” (Week 4 reading list). The chapter introduce the integration framework to help you see integration happening not only at the technical level but at the process and the people level. Hence integration happens at both the process and the structure.
7
Let get you started with looking at the 7 databases as 7 separate entities which is how Integrated Trading is now operating. See diagram below…
APP
APP
DB
DB
APP DB
APP DB
APP APP
APP
DB
DB
DB Although the icon represents Tight coupling of the application and the database, the representation includes both tightly coupled (legacy, monolithic apps) and loosely coupled (modern, modular apps)
Figure 4
There are indeed many ways one would interconnect with one or all of the 7 apps/databases
APP
APP
DB
DB APP
APP
DB
DB
APP
APP
Figure 5
DB
APP
DB
DB 8
The above connections whether they are physical or logical connections, there are integration mechanisms that need to be introduced to enable such integrations. Don’t forget that all options have their tradeoffs and some options will violate the constraint of not forcing all applications to be changed. Let’s explore various different levels of integration that you should consider. At the Data level
APP
APP
DB
DB APP
APP
DB Integrator/ Integration Mechanisms
DB
DB DB
APP
DB
APP
APP
How about at the Application level….
Integrating At the APP Level – Would it force applications to be changed?
Integrator/ Integration Mechanisms
APP
APP
APP
APP
APP
APP
APP
DB
DB
DB
DB
DB
DB
DB 9
How about at the people and business process level….
CR
CR
CR
CR
CR
CR
APP
APP
APP
DB
DB
DB
CR
DB
DB
DB
APP
APP
APP
CR CR
CR
CR
APP DB
CR
CR
CR
This will get you started to think about the possibilities in your integration design alternatives. Before you start sketching away the STRUCTURE of your ideas you need to evaluate your design alternatives with first and foremost the constraint of not forcing all applications to be rewritten, but you should also consider the following points. You should treat these 7 separate businesses as being significantly different in the product and the required knowledge to service the customer. A common customer rep would not be a good option – each area need fairly specialized knowledge to serve the respective client. However, the existing customer rep will indeed be able to get up to speed on the general product offerings in other business units. One of the many reasons is that the relationship (often well established) between the customer and the customer rep will tend to make that customer rep the primary point of contact/interface. You CANNOT change the code but you could add additional modules or code to the system
10
The guidance below is useful to you after you have had a chance to develop a number of solution alternatives. Review these key points as you think through the alternatives and weigh the tradeoffs among them. 1. Finding the balance between how much they should be or can be brought together physically vs keeping them separate and use integration mechanisms to synchronize them is key 2. As you consider various options, you want to make sure that your chosen alternative can withstand the test of time What if additional business units are acquired in the future What if one or more of the units are sold off and hence will decoupled from the integration design 3. Since each business unit have products that are fairly specialized, they certainly would have their unique set of requirements Performance and scalability Security/privacy requirements (regulatory in nature) Politics between and among these business units How would your design accommodate that?
Exercise: Sketch or draw how you see the 7 databases and their associated applications will be organized/arranged (STRUCTURE) to enable the integration of these 7 areas thereby addressed the underlying problems the Integrated Trading has been experiencing. Provide at least 2 alternatives, 3 is better.
11
Exercise: Show in the form of a flowchart or sequence diagram to demonstrate HOW (the process) customer information are integrated as they are being added, change and or deleted from the integrated system. Think of integration mechanisms from the standpoint of whether integration is via synchronous or asynchronous communications. The key is the mechanism and the process to show HOW data/information are synchronized.
Once you have layout the idea both in STRUCTURE and PROCESS, it is time to revisit the scenario narratives that you had captured earlier in the problem space to find out how the interactions between the customers and the customer representatives might be different and how the design alternatives address the problem that you had identified. Exercise: Show the scenario narratives here….
12
To start you off, here are a few scenarios that you should definitely show. 1. A customer calls into the owners of one database, and wants to know information from yesterday stored in a different database. 2. Suppose a new customer rep in Integrated Trading wants to see all the information about an existing customer across all services. Will your design allow for that? Where would the customer rep interface with the database or databases? 3. How would changes to the information in one database find their way to update other databases that have the same customer information?
Scenario narratives allow us to capture the way things are done, the problems encountered, challenges noted and not the least the outcome achieved or not achieved. It is not enough to get rid of the problem encountered that prevented us from achieving the goals and objectives. Scenarios have the capability of being used to play out situations that has not been possible as the existing STRUCTURE and PROCESS are preventing that from being carried out and achieved. Play out other not yet thought of scenarios that would benefit Integrated Trading and use those scenarios to further help your design endeavor, making your design alternatives capable of supporting and responding to how Integrated Trading as a business (the containing system) adapts to the customers and business in general. If you had followed along and produced the scenario narratives, diagrams (structure of your problem and solution space) and flowcharts/sequence diagram(Process of your problem and solution space), you will be ready to use these illustration to create your poster presentation.
13
