The BASIS of SAP

Install SAP on Amazon Web Services #2 – the Installation

After my previous post, you either have your own Windows 64-bit AMI image, or access to the Public AMI I have created, called sap.nw70.win-64.db2. In this exercise, we will use this as the basis of a new, private, image that will:
* contain the appropriate installation data (including registery keys) for SAP NW7,
* be capable of online / offline backups, using SAP tools,
* provide a painless way of running 24×7.

Prerequisites

* EC2 and S3 Accounts with Amazon,
* access to a Solution Manager system (for the installation key),
* access to an OSS ID with download authorisation.

Architechture

Once we terminate an Amazon instance, we lose all changes to it. Saving our database and configuration changes by bundling the changed system into a new AMI will take a non trivial amount of time; Certainly enough to prevent it being run 24×7. Additionally we will lose lots of usefull ABAP and JAVA stack logs unless we bundle the running instance every time we shut it down.

Just as well there’s an alternative, called Elastic Block Storage. This allows you to create data volumes and mount them on your image. They are persistent, and more importantly, can be backed up by snapshots, from the AWS Management Console.

So that leads to an architechture (or rather, disk layout) as follows:

• Drive C: AMI instance, boot disk
• Drive D: AMI instance, ephemeral disk (data lost whenever instance shuts down)
• …
• Drive H: AMI instance, ephemeral disk (data lost whenever instance shuts down)
• Drive W: Persistent Disk, for storing disk-to disk backups DBMS and / or logs
• Drive X: Persistent Disk, for SAP and DB2 Intallation
• Drive Y: Persistent Disk, for DB2 logs
• Drive Z: Persistent Disk, for storing installation files

Creating EBS (Persistent) Volumes

To create EBS Volumes, go to the EBS Volumes section of the Amazon Management Console. The major issue with creating volumes is that you can only attach / mount an EBS volume on an instance that is running in the same Availability Zone. This does mean that all your volumes must be in the same Availability Zone, if they are to be attached to the same instance.

I’ve created four volumes, corresponding to the Drive Letteres I gave in the Architechture section above.

• Drive W: vol-a82bc7c1, for storing disk-to disk backups DBMS and / or logs
• Drive X: vol-3f658956, for SAP and DB2 Intallation
• Drive Y: vol-4451bc2d, for DB2 logs
• Drive Z: vol-fc2bcb95, for storing installation files

Note that these are empty, unformatted, unmounted, unattached volumes (at the moment…).

Attaching EBS Volumes to our Instance

To attach the volumes to an instance, we need to have an instance running. Start up an instance of your image or of sap.nw70.win-64.db2.

Note that I am creating an x.large instance in the availability zone US-east-1b. I need the x.large instance to provide enough RAM and Swap Space for an IDES ECC6 system, and I’m starting it in the US-east-1b availability zone because thats where I located my volumes (no particular reason).

Once the instance is running, we can attach our volumes via the Attach Volume Button.

The result is that our volumes are now “physically” attached to our instance. Again, these are empty unformatted unmounted volumes.

Now we need to logon to this instance. If you are running an instance of sap.nw70.win-64.db2, you can logon as user sapinstall, password sap123. Use the Remote Desktop Connection, and specify the public dns name from your instance.

You assign a name to a volume when you are formatting it. You do this by running the Computer Management (if you’re running an instance of sap.nw70.win-64.db2, this should be on the Desktop of user sapinstall) and formatting and naming the volumes. Make the names distinctive, and related to their purpose, for example sw_repository.

Now use the C:\Program Files (x86)\Amazon\Ec2ConfigSetup\Ec2ConfigServiceSettings.exe program and the Drive Mapping tab to control which volume gets mounted to which drive letter. This is important, because we want to make sure that our sap_install, db2_logs, and backups volumes are always mounted on the same drives. Once the current image is bundled and registered, any instance launched from the new AMI will contain the setting we have configured in Ec2ConfigServiceSettings.exe.

Note the relationship between the volumes and Drive letters in the image below compared to the description of each volume given in the Architecture description above.

System Specific Configuration

Change the hostname (or in Windows terms, the Computer Name) to one of your choosing (Start –> Control Panel — System –> Computer Name –> Change). Run Ec2ConfigServiceSettings.exe. and make sure the Set Computer Name flag and the Sysprep flag on the Syprep tab are disabled – They should already be disabled, if you are using a copy of sap.nw70.win-64.db2.

Check the swap space (Start –> Control Panel — System –> Advanced –> Performance Settings — Advanced, Virtual memory). Again, this should already be correctly set if you are using a copy of sap.nw70.win-64.db2.

Edit the hosts file in C:\windows\system32\drivers\etc to include your Computer Name as a valid host name, for internal SAP and DBMS connectivity.

Do not forget to change the password of the sapinstall user. Otherwise, anyone who reads this will know the password.

Finally, bundle the instance using the AWS Management Console and register the resulting image under your own image name. The purpose here is to save the customisation you have done if you have a problem with the SAP installation. As part of the process of bundling, the instance is shut down and restarted.

You do need to have an S3 Bucket (or directory) to store the Image in.
However, you can store multiple images in the same bucket, by varying the Amzon S3 Key Name.

For future reference, if you restart the instance yourself, using Start –> Shutdown and specifying Restart, you don’t loose any information or configuration from the C drive as you would if you terminated it from the AWS Management Console. This is because the later removes the underlying resources, while using Start –> Shutdown –> Restart doesn’t release the underlying resources.

Security and Firewalls

EC2 provides its own set of firewall rules called Security Groups. The defaults values are, essentially, just enough to get you access to the server itself.

Since SAP communicates via TCP/IP, we need to make sure that our instance(s) can be accessed via the ports used by SAP for its various services. This means we need to add the ABAP and Java ports for both our instance and the diagnostic instance.

Remember that the Windows Server underlying your new SAP system is on the Internet, and is accessible (by Design !!) from anywhere else on the internet, so only open the bare minimum of ports.

Installation

Download the appropriate files from http://service.sap.com/swdc (you’ll need an S number with download authorisation), extract / expand them and store the results on the Z drive. I stored the download files under Z:\NW70SR3 and expaneded them into their own folders on the Z drive.

Make sure you read the appropriate OSS notes. For the ECC6 IDES, the important ones are:
0799639 – General IDES related
0956921 – NW7 ECC6 SR3 IDES related
1244548 – NW7 ECC6 SR3 IDES related
and
1126127 – DB6: Deferred Table Creation and Row Compression

Otherwise, the install follows the standard process, as detailed in the appropriate installation guide (in my case, the NW7.0 SR3 ABAP+JAVA / Windows/ DB2). The two exceptions are:
* Specify that the SAP and DBMS Installations go on an EBS volume (i.e drive X)
* in my case, specify that the DB2 logs go on an EBS volume (i.e. drive Y)

The full IDES install took around 30 hours run time (think of it as $20 or so well spent) from when I started sapinst (that time did include checking and amending my previous implementation notes). The majority of the time is spent loading about 150GB data into the DB2 database. However, once sapinst had accepted the Solution Manager Key, you can disconnect RDP and leave the install running.

Saving your image

Once the installation is complete, you’ll want to back it up before you go any further. Using the SAP MMC, shut down SAP (or logon to Windows as the SAPService<sid> user and shut down SAP).

Use the AWS Management Console to bundle your running instance.

Once it is bundled, register the bundle as an instance.

You can share this with anyone with an EC2 account, by using Permissions to mark it Public, or you can share with individuals if you know their EC2 Account number. Note – Bundling a windows instance restarts the instance.

Basically, the image consists of whats on the C Drive, so backing up your EBS Volumes requires you to use the AWS Management Console to save snapshots of them. The EBS volumes are stored and charged for at the Amazon S3 rates. Just like EC2, however, you are only charged fo what you use. This means that if you define a 500GB volume, write a 1 GB file to it and create 4 snapshots of the volume, you will charged for 5GB of storage; 1GB data on the volume, plus 4 lots of 1GB of snapshot. backup.

When you’re finished with the instance, shut down SAP and don’t forget to terminate tthe instance via the AWS Management Console (otherwise you’ll be charged for it !!).

Running your SAPSystem

Start an instance of your image and attach the EBS volumes to the running instance. The work of of assigning drive letters, in the correct order, to each volume is controlled by our configuration work earlier in Attaching EBS Volumes to our Instance. One of the issues currently outstanding is that thess will actually get mounted on subsequent restarts of this instance (which we perform below).

Logon to the instance and update / verify the Swap Space sttings via Start –> Control Panel — System –> Advanced –> Performance Settings — Advanced, Virtual memory.

Regardless of the previous paragraph, restart the image using Start –> Shutdown -> Restart. With all Drives correctly assigned, and sufficient Swap Space assigned the DB2 and SAP Services for SAP MMC will start. Go into SAP MMC and start your SAP instance. Once SAP is running, you can disconnect from the instance.

Accessing your SAP System

Assuming you have opened the correct ports in the Security Group specified for this instance, you can now put the appropriate values into your SAP GUI …..

…..and access the ABAP Engine.

Again assuming you have opened the correct ports in the Security Group specified for this instance, you can go into the SMICM transaction and enable a simple service, then access it via a browser or web service.

Whats next ?

You now have a running SAP system. However

• No DBA processing, i.e. no DB13 jobs, no backing up of logfiles etc has
  been implemented, so once you’ve tested connectivity, stop the SAP and
  DBMS systems and take snapshots of your SAP & Database volume.
• The SAP*, DDIC and IDADMIN passowrds are well known (or easily determined). Change them
• No post implementation work (i.e. SGEN) has been done,

The purpose of the exercise is to demonstrate how quickly you can run up a demonstration, training or testing system. Depending on how many resources you want to pay for (CPUs and memory), this can be quicker or slower.

However, it has been my experience, based on several green fields implementations, individual system implementations and upgrades, and feedback from others, that building an appropriate server – whether physical or virtual – can take up to 2 weeks. Using the approach detailed here, services such as provided by the Amazon EC2 service reduce this to the 45 minutes it takes to configure and bundle a standard public instance.

One of the obvious issues is that it is well and good using predefined data, which you can download, in zipped form, from OSS (such as the IDES data I used in this example). What about copying ‘real’ data fron an existing SAP system, especially if we’re talking TerraBytes ?

I’ll discuss this, the bandwidth of a portable hard disk and more of the Amazon Web Services features that are particularly useful for SAP in my next post.

Install SAP on Amazon Web Services #1 – The Environment

UPDATE: I have tidied this up a bit, to make some things clearer and to include the name of an AWS Public Image that can be used as the source for the subsequent step.

In this post, I describe how I setup a windows environment to install SAP ABAP and Java stacks, using the Amazon Simple Storage Service (S3) to store persistent data.  I needed to:
* install and modify an appropriate Windows 2003 Server environment,
* save this environment for future use

In a subsequent post, I will describe the installation of an IDES system running NW7 and DB2.  The three major challenges were
* setting up persistent storage of the NW and DB2 installation,
* suitable for using standard SAP and AWS functionality to support sustained (i.e. 24×7) operation of the SAP system
* and allowing you to stop and start the SAP system and / or server without losss of persistent data.

The result is a fast and cheap way of running up multiple systems, with the following features:

* You are only charged running costs for those systems that are running
* Low running costs (at the time of writing, $US 50 cents an hour)
* Low storage costs ($US 15 cents / GB / month for your 50TB)
* No more waiting for hardware – you can start implementation right now
* Systems (i.e. extra application servers) can be implemented, but not running

 

What did I know I would need ?

After reading the NW 70 SR3 installation Guide for Windows / DB2, I knew the following:
* I needed a 64 bit Windows Server with authentication services,
* I needed a reasonable amount of RAM, plus a decent swap space,
* I needed JAVA 1.4.

After reading the AWS EC2 documentation, I also knew that it was not practical to keep any volatile datasets (i.e. DB2 itself, DB2 logs, SAP process logs, etc) as part of the server, and that I would need to use the Amazon EBS servcie for persistent storage.

 

Signing up for Amazon EC2 and S3

An excellent account of how to setup a Windows Server image, and the principles behind this, can be found at Dave Winer’s EC2 for Poets. It also gives a good overview of how to sign up for both EC2 and S3 and the issues around persistent data.

 

Creating the base Amazon Machine Image (AMI)

Logon to the AWS Management Console and select the Amazon EC2 tab.

Select the Launch Instance button…

.. then find and select the Basic 64-bit Microsoft Windows Server 2003 with Authentication Services image.

 Once the server shows up as running, logon using the techniques described in Dave Winer’s EC2 for Poets.  One of the first things I did was to create a sapinstall user.  This allows me to logon (via RDP) as user sapinstall / password without having to muck around with the keypairs. 

 

Changes to standard AWS Windows 2003 64-bit Image

There were five issues that needed to be dealt with.

First I had to disable the Windows Attachment Manager (for non-windows people, this is a security setting that Windows uses to stop you writing dangerous file types to your disk) before Internet Explorer would let me save files.  See the Microsoft Knowledge Base Article 883260 for a rundown on how it works.  The quickest way to disable it is to uninstall the Internet Explorer Enhanced Security Configuration. To do this, click Add or remove programs in Control Panel, click Add/Remove Windows Components, and then click to clear the Internet Explorer Enhanced Security Configuration check box.

2) Both SAP and DB/2 (my target DBMS) require that the host name of the server its installed and running on remains the same.  However, the default action every time you restart an AWS image is to have the host name set to IP-xxxxxx where xxxxxx represents the internal (to Amazon) host name the server is running on.

While you can perform arcane scripting to fix the host name, Amazon provide a tool, bundled within every AWS windows instance, that will ensure the hostname remains set to what ever you set in the System –&gt; properties screen.   The tool is C:\Program Files (x86)\Amazon\Ec2ConfigSetup\Ec2ConfigServiceSettings.exe

 

3) I wanted to make sure I had enough swap spacxe to run my SAP system.  The base instance we are using gives us 15GB of memory, but, especially if we want to install multiple JAVA engines, this may not be enough.  I allocated another 1500MB on each of two of the ephemeral disks.

4) My initial installation is going to be an NetWeaver 7 ECC6 system. This means we need to download and install java 1.4 from Sun’s Sekrit Squirrell place for old releases.  Don’t forget to setup the Environment variables (JAVA_HOME and PATH) correctly. 

5) The last change was to incorporate a Dynamic DNS Update tool.  This is used to pass the IP address of the server we are “running on” to a service that will then set a fixed Domain name to specify the same DNS name to users and tools whenever I ran my instance.  I use dyndns org.  You can register a limited number of domain names for free, and they provide a tool (DynDNS Updater) that allows you to register your IP address against one or more of your Domain names.

 

Save your Amazon Machine Image (AMI)

Now you have an instance you can use to install and run SAP on.  However, we need to make sure that all our changes are not lost.  This utdown means you need to “bundle” your running system into a standalone Amazon Machine Image.  Go to the Amazoin EC2 tab of the Amazon Management Console, select Instances, then select the instance you want bundled.  Right click on More Actions and select Bundle Windows AMI.

This generates a popup screen.  Fill out the appropriate details and clcik bundle. The Bundle Name refers to the S3 folder that will hold the AMI.  This must already exist.  The Key Name is appended to the name of manifest.xml filre that contains the S3 layout and location of your image.

 Once you click bundle your request is confirmed.

.

You can follow the progresss of the bundling by examining the Bundle Tasks screen.  There are three steps that bundling Windows instances needs to follow- The instance must shutdown, the Amazon bundling process must occur, and the resulting data must be stored.

Once the image has been bundled and stored, you must register the bundle as an Amazon machine Image.

 

An alternative to repeating all the work shown above is to grab a copy of the Public AMI I have created, called sap.nw70.win-64.db2.  You will need to change the hostname (as descibed above), implement your own DynDNS org domain name  and bundle and register the changed image.

Either way, you now have your own mildly customised image copy of a Windows 2003 Server, running on the  Amazon Web Services cloud.  This image is ready for installation of a non-trivial SAP system, such as the NW7 ECC6 IDES system.  

In the next post, I will describe how I used the sap.nw70.win-64.db2 image to install the Windows DB2 IDES for ECC6 system.

2 ways to Measure Exact Throughput of a TCP IP network

One of the sizing issues with an SAP system that doesn’t receive due consideration is the network capability; not just speed, but throughput. It’s always usefull to know what your Network is capable of, especially if you have lots of data to move (Support Packs / Support Stacks and so on). But how do we find out?

NetCPS (a single executable file) is rather simplistic, with no fancy features as the author (credits to Jarle Aase) says. It pumps 100MB of generated data (without accessing the HDD which could mess with the final result) and then displays the result in form of average speed stated in both KB/s and MB/s. You can also get source code if you’d like to do some further tinkering with it, or port it. Everything you need to know is on the webpage or available by using -help switch.

Another, more sophisticated, tool (without being too big) is Iperf (a single executable, with source available on the same page). Settings are changed by use of various switches.

For example, the image above shows the port used is changed to 1234, amount of sent data set to 200 MB, interval of reports set to 2 seconds for better accuracy and report format set to MBytes. The usual -help switch brings up further instructions for changing the many additional switches and settings available with this tool

Network tools for BASIS Administrators

Depending on how advanced the rest of your IT organisation is, you may need to be the jack of all trades. In fact, sometimes it feels as if anything that a Developer or End User doesn’t understand automatically becomes the property of the BASIS Administrator. Typicaly, these can include anything to do with the infrastructure between the users desktop and the SAP application.

An example I’ve been currently working on is a network issue where a user can access the Portal from one machine but not another. I used to use separate tools to do my network monitoring and debugging (yes, there are people responsible for this, but I have won a lot of good will by providing as much data as possible), but these days ….

Net Tools 2008 has been described as The Swiss army tool for network administrators everywhere. It is a very versatile tool, and
just like any tool it can be used for good or evil. What this means is that you may find the site blocked at work.

Available functions, usefull for both the desktop and Windows Servers, include

An FTP Client for quick file transfers,
Monitor system up status with Monitor Host IP,
Mass file renamer, to rename a whole bunch of log files,
Bandwidth Monitor.

Another tool that I’ve feard of is eToolz which is a collection of network and Internet tools that provides a graphical interface for several common commands. This includes ping, tracert, DNS lookups, http headers, default ports, etcetera. This seems more directedt to someone who who supports web sites, but after all thats what the Portal is …

SAP Windows : Monitoring Disk usage

This will be useful for any SAP system running on Windows, or for your own desktop or laptop.

DriveSpacio is a free Windows utility for examining hard drive usage. When you first boot the program you’ll get a list of each hard drive and partition on your system, along with, some details like the files system, cluster size, and a pie chart or bar graph showing used and free space.

But the fun really begins when you click on the Folders section. You can choose a folder, or an entire drive (just click C: or E: or what have you in the browse window), and DriveSpacio will plot your folders on a graph showing you which of your folders are eating the most space. It takes a few moments to scan folders with a lot of subfolders or files, but the result is a pretty effective tool for figuring out why you’re running out of space on your hard drive.

WinDirStat is another free utility which does pretty much the same thing. But if you prefer the bar graph/pie chart look to WinDirStat’s more abstract-looking visualization, DriveSpacio is worth a look. But there is at least one area where WinDirStat holds the edge: while you can delete files from the WinDirStat interface, DriveSpacio only shows you file and folder names. It doesn’t let you launch or delete files.

 

From Shell Extension City