Alongside the normal challenges of increasing data, IT administrators in the health sector are faced with a number of industry specific challenges. For example, critical patient data such as X-ray images are being increasingly saved in digital form. Moreover, these images are dramatically increasing the demand to add more data storage and the new imaging technologies and their number and use continues to grow daily, thus exacerbating the problem. The hospital information system KIS also plays a central role in hospital management, patient records and data access. Medical data is critical and has to be accessible 24 hours a day. This means that the highest levels of availability of data are a mandatory requirement, meanwhile healthcare budgets are not keeping pace and therefore cost pressures are now at an all-time high. Even in an environment where spending is tight, the IT requirements are constantly growing and changing.
In 2003, the Schwabing hospital opted to move its Direct Attached Storage (DAS) to a Storage Area Network (SAN) so that the data storage resources could be more easily added and connected to more systems. The hospital needed to take this step at that time because of the switch to a different laboratory system and the consequent bottleneck it was seeing in capacity. The hospital needed to decide whether to set up another RAID array system or whether to free up disk capacity from the directly attached, dedicated systems by implementing a storage network. Another reason was that IT administrators wanted to consolidate and virtualize a portion of their system landscape using VMware?s ESX-Server software as part of their plans in the medium term. To do so, they knew that VMware best practices dictated that a SAN infrastructure would be a necessary requirement.
?When choosing the SAN solution, the overall costs were decisive for us,? explained Kosmas Schütz, the chief IT administrator at the Schwabing Hospital in Munich at that time, who is now in charge of the IT infrastructure across the entire multi-hospital network. ?We wanted to move away from hardware and supplier constraints and provide ourselves with the opportunity to expand storage as we needed in a linear fashion. This is only possible with a SAN, which virtualizes and centralizes storage management and storage services. After taking a good look at the market, we were certain that a software-based virtualization solution was what we needed. But it was still a big step for us because the technology at that time was new and was not as established as it is today.?
Storage Virtualization
After an extensive, six-month selection process, the Munich project was awarded to CEMA AG, who impressed the Schwabing Hospital with their implementation design, which featured DataCore SANsymphony software. CEMA installed the Fibre Channel SAN infrastructure and storage in just four weeks at the beginning of 2004. Two Dell Servers running Windows 2003 were transformed into storage servers using SANsymphony and they formed the basis of a dual high-availability architecture, which was connected to two redundant Silkworm 3800 Fibre Channel-switches from Brocade.
As Storage Domain Servers (SDS), they control all the storage accesses and allow any storage to be allocated from a central console. With 2.8 GHz frequency, two dual processors and 4 GB RAM used as cache for I/O acceleration, the SDSs are extremely high-performance. DataCore?s auto provisioning function (?Network Managed Volumes?) automatically allocates and optimizes storage use since it only provides storage when it is used and needed by systems ? meaning that large virtual volumes of up to 2 TB each can be served to the applications so that they can continue to operate without running out of capacity. And the actual storage for the site can be less since it is part of the physical storage pool that can be managed and shared across many systems. DataCore?s mirroring technology, which is transparent to the applications, allows synchronous replication of all data between the two SDSs so that if one system fails, all the data is available over a different path and production continues to run without impact. The actual disk subsystems themselves are also protected by RAID-5 in case any single disk fails. The Snapshot function, which creates chronological copies of datasets, was also licensed by the hospital. These point-in-time snapshot copies can be used, for example, to perform off-line analysis, to run tests and to simplify disk backups and migrations.
Continuous Expansion of Capacity
The flexibility of the solution was first demonstrated with ease in the integration of additional disk capacity. Since 1TB of capacity was sufficient to demonstrate the first implementation phase, the hospital decided not to invest in expensive FC arrays. Instead, they decided to integrate more cost-effective SCSI disks into the SDSs themselves and to use these as SAN resources. Administrators were also shown that it was possible to re-use these disks when the storage pool would be extended in the future. According to Tobias Wellhausen, Vice President of CEMA AG, "This was a simple way to demonstrate the flexibility of a virtual, open solution and the advantages of hardware independence. Together with the team at Schwabing, we were able to implement a tailored, entry-level model into the world of SAN, which the hospital was able to use and learn from and which they profited from right from the start. Meanwhile it has evolved into a comprehensive infrastructure platform to meet their medium and long term needs.".
In a second step, the hospital?s IT team planned to integrate two local EMC Clarion 4700 Disk Arrays into the SAN, while at the same time the other hospitals were inquiring about expanding their own EMC systems. Since the project had demonstrated the value of the consolidation plan and the benefits had already been realized, there was no point in trying to deploy new subsystems just to meet the needs for a short period of time. Therefore, an unusual solution was necessary.
The EMC subsystems were quickly evaluated and it was agreed that they should be deployed within the existing IT environment. This added capacity helped solve the immediate lack of disk capacity, bottleneck problems. In the Schwabing Hospital, network administrators installed additional costeffective FC disk subsystems from Nstor (4500-Series FC/FC). They also later added 5200-Series arrays with SATA disks. "This meant that the storage network was able to absorb new technologies and did not require an overhaul or new systems,? explained Schütz. ?The advantage for us was that we were free to choose which disks we wanted to add and therefore the price of the disks that we decided to implement turned out to be the same price as the maintenance cost we had paid for the previous system that we had installed. This represented a dramatic cost savings and a classic win-win situation."
Storage Classes and Different Levels of Quality of Service (QoS) Support
Over the course of time, additional applications were integrated into the SAN, and the SAN was extended to 14 TBs. The SDSs, which were initially fitted with 4 FC ports, had to have two additional FC ports added in order to provide the new applications with enough performance. The SAN was therefore configured into three zones, each with 2 ports (path redundancy). Two ports provide the intercommunications between the Storage Domain Servers for data mirroring and failover. They also link the SDSs to all the storage subsystems. Another two paths lead to the application servers.
The SANsymphony SDSs also improved overall performance. DataCore's special caching algorithms and I/O acceleration technologies gave the overall SAN system a new level of high performance. When measuring performance, the resulting values were similar or better to what were seen on highperformance RAID system arrays, but these results were achieved at a much lower price using less costly disk technologies. The 60 MBs of I/O throughput that the Schwabing Hospital needed was dealt with easily.
However, a central solution also has its problems, as was shown when the hospital tried to integrate into the SAN a slower, used disk subsystem that was available as mirrored storage managed by SANsymphony. Under certain data loads, the system experienced performance problems, which were attributed to the age of the disk subsystem deployed. Since the mirroring was applied to the very slow disk storage, it became the slowest point and bottlenecked the "max speed." As a result, the overall system became too slow and that live SAP systems, which were using the storage, also experienced performance problems. When a new, more modern target for the mirror storage was defined and the obsolete system was removed from the SAN, these problems no longer occurred. This proved that with SANsymphony, users can utilize almost any storage hardware. However, if performance is an issue you have to be careful that the hardware can handle the basic load.
After continual expansions, the Schwabing Hospital now has data storage functionality and capacities of varying types, performance levels and quality, which are used for different purposes depending on the application need and requirement. The more expensive, high-performing FC Systems (FC interfaces, FC disks) are used for "mission critical" applications, such as SAP. The laboratory system runs under FC/SCSI. FC/SATA disks are used for other applications with lesser demands.
"This was a fundamental decision,? noted Schütz. ?We get around the possible higher risk of system failure from cheaper hardware by implementing a fully redundant system. The ability to support different classes of storage and the ability to deploy a completely redundant storage architecture enables protection and a choice of hardware ? heterogeneous storage managed by SANsymphony. Interestingly, this is still cheaper than a single, high-end solution from other providers and it protects our data just as well, if not better."
Various Applications on the SAN: Centralized Storage Management
Munich-Schwabing is now providing the central infrastructure for servers and storage in the integration project for the five hospitals. The first of these centralized applications to be served capacity from the SAN was the hospital?s laboratory application. It is run on redundant IBM pSeries servers, which use storage from the SAN and in fact are themselves booted directly from the SAN ? adding another level of data protection to the boot disks. Alternate Pathing (AP) software from DataCore also offers additional availability for the servers? connection to the SAN and the software re-routes the connection to an alternative path if a data path fails.
A problem was identified concerning the original IBM AP software, which was addressed quickly by CEMA. The AP software of the server vendor only worked with the vendor?s own storage systems. CEMA worked closely with DataCore to solve the problem and found that DataCore's driver for Alternate Pathing in the AIX-environment worked equally well. This was installed for live operation after tests had been carried out.
During the consolidation of the five computer centers, additional server hardware was required and had to be integrated into the SAN, plus additional disk capacity was now critically needed. SAP R/3 applications were running on Sparc servers under Sun Solaris as well as on Opteron servers under Linux. Blade Servers with MS Cluster for Windows 2003 were deployed for central Filer, Exchange 2003 and SQL 2005 Server applications. "We wanted to make it easier to manage, and therefore we set up a single and central storage network using DataCore SANsymphony for as many of our systems as possible," said Schütz.
Server virtualization
The heterogeneity and scope of the applications and hardware environments made it logical to go beyond storage virtualization to go to the next step with server virtualization. In Munich Schwabing Hospital, the introduction of a VMware ESX Server had been planned and anticipated for a long time.
The VMware software today runs on two high-performance Dell 6650 servers, which offer 16 GB of main memory and four 2.5 GHz Xeon processors. Another Dell 6650 Server is used for both back-ups and for testing ? for example, to test for compatibility problems with new software releases. The VMware servers are connected to the SAN. The virtual platform is divided into a total of 70 logical servers. Between 30 and 40 of these virtual units are only temporarily in use for test purposes. If necessary, a virtual machine can be set up for each application and each area of use, which increases the stability of the individual servers. If servers within the server farm are switched to VMware software running in live operation, the working system and storage space is copied to a different location. Obviously, with many different virtual servers running on these two platforms, it is critical to protect the underlying storage and this requires a robust SAN infrastructure.
Cost-efficiency is also a top priority when embracing server virtualization. Each VMware platform replaces a hardware system and each would need to be equipped with two FC ports. The costs alone for physical servers, FC and the storage requirements for the corresponding hardware could easily exceed the budget of a hospital. According to Schütz, ?VMware works for servers like DataCore?s SANsymphony works for storage. We can now set up and implement computers very quickly, whether for simulation and test purposes or for live operation." Even though the hospital had a clear budget at the beginning of the year, there was no definite information about how many servers, applications and how much storage it would have to order during the course of a year. This meant that it was extremely difficult to calculate total requirements. Added Schütz, "In IT today, you need to be able to provide infrastructures very quickly. This is only possible with virtual infrastructures like the ones we now have."
This new flexibility has been used within the hospital to introduce Quality of Service and to optimize the load on existing systems. If smaller systems are required, which is normally the case, a virtual system is set up using VMware. If these smaller systems exceed the capacity and performance requirements of VMware, the application is either switched to the Blade Server, or, if absolutely needed for some cases, a dedicated server is then set up.
When operating system updates or hardware upgrades are needed, the cost and time savings are not the only advantages. In live operation within a virtual environment that is under the control of SANsymphony and VMware, it is now possible to upgrade the server hardware and provision storage relatively quickly. This is in sharp contrast to the complex, cumbersome and very time consuming scenario that would be the case with 70 physically separated machines and the corresponding storage allocation required for each machine.
Training and Administration
Two employees were trained by CEMA to learn and operate the VMware environment and the SANsymphony environment. According to Schütz, handling and operating the software was not a problem after the training they received. In fact, it was just the opposite. "The employees very quickly got used to the features and they really liked the responsiveness and the ability to quickly handle new requests,? explained Schütz. ?But in centralized systems, security, availability and guarding against system failure and downtime is the absolute requirement. A local server with a dedicated application can fail at any time. However, this cannot happen with our consolidated solutions and the flexibility of our virtual environment! The consolidation project was therefore a learning process for our staff that has opened up a new way to do IT ? adding more flexibility, faster response times along with new ways to protect systems and storage using high-availability virtual infrastructures and SANs."
In terms of utilization, maintenance and administration of servers and storage, Schütz estimates that the use of virtual infrastructure improved overall resource utilization by at least 10% ? and possibly as much as 30%.This is one of the primary reasons why the rest of the Munich hospitals decided to consolidate their IT operations and to integrate them into the virtual architecture of the Schwabing Hospital. The Schwabing team now has the right architecture in place as well as the know-how and the expertise to make sure that the system can be integrated easily and runs smoothly, now and into the future, from a central location. "Our virtual SAN solution fully meets our expectations. We have attained mainframe features at Intel server prices and are afforded excellent support day and night," concluded Schütz.
 ShareThis
Tags: Applications, Hosting & Colocation, Design & Facilities Management, Networks & Telco , Security, Servers, Storage Networking |
