sap integration suite cloud cpi

SAP Integration Suite Cloud CPI - Create your first SAP Integration process

Below is the basic tutorial that will show you how to implement your first simple SFTP Integration process. This will allow you to import a file from one location then perform any transformation on it via Croovy Script then send the resultant file to a second location. Being an expert by the end might be pushing it but I feel it shows you all the basics you need to know and you will be able to work out anything else with a bit of help from our friend google.


Step 1 - Login into a SAP integration Suite/CPI
These first steps are a little fidily to explain, especially as im going to introduce the old SAP CPI version and the new SAP integration Suite version. But it's basic stuff and once done you will know everything you need to know about SFTP connections within a CPI/SAP integration suite and will have a basic platform to build your CPI knowledge.


Monitor Message Processing
There are two main screens when developing SAP Cloud Integrations, the Monitor message processing screen and the Design Screen. Initially we will focus on the Monitor screen.

Within SAP integration Suite you will land on a page like this




Expand the graph looking icon and select Integrations




and you will be taken to the Monitor Screen





Within SAP Cloud integratiuon / CPI you will land on a page like this




Simply Click on the eye icon to view the Monitor screen








Step 2 - Setup an SFTP connection
I think this next step totally demistifies the SAP integration suite

Choose the Connectivity Tests Tile




Now enter your FTP server details. You may get a hosts error, which means you need to add the host key to your known_hosts file




To do this simply check the "Enable Support for Deprecated Algorithms" and press send again and it should provide you with a key...




Now Click the "Copy Host Key" button and the key will be copied to clipboard




Now you need to create a known_hosts file on your local machine, and paste this key data into it. Ensure the file has no extensiomn i.e. remove the .txt




Return to the overview screen and select Security Material tile




Now click the Upload dropdown and select Known Host (SSH)




Now use browse to find the known_hosts file you created, click deploy




An new SSH Know host entry will be added to the security material list




Step 3 - SFTP user credentials
Now you need to create a user credentials security material entry. Again within the "Security Material" tile but this time click the Create dropdown option and select "User Credentials" or whatever you authentication method is. Hopefully you will starting to see how easy this could be to setup, even if you are using other methods such as OAuth etc..





Enter you SFTP login details that you want to use to login to the SFTP/SSH server.




An new entry will be added to the security material list




Now return to the Connectivity Tests Tile to test your user credentials




Enter all the details including ftp server, check "Enable Support for Deprecated Algorithms", select your Credential name, and select "Check Directory Access" just so you can see the directory list coming back.




You have now setup an SFTP connection!!!



Step 4 - Build basic SFTP CPI integration.

First step is to enter the Design screen via the pencil icon and choose or create a package. A package is just like a folder where you build all you related integrations. These individual intergrations are called Artifacts.





To create a package simply click create and enter your package details




Once you are within you package and in edit mode




you need to create a new integration artifact




Enter the artifact details




The new integration will then be created




click on it to enter the design screen and click the edit button





Step 5 - SFTP to retriev the intial file

The first step of the build process is to link the sender to the start intergration flow,




to do this, drag the connector link from the sendor to the start.




then select SFTP




Now look to the footer of the page and drag the slider up to see the SFTP config details, select the Source tab




Enter the directory and file name pattern you want the integration to check for, along with your FTP details, including the user credentials enty you created earlier(note there is no dropdown for some reason) so just enter the name your created. Also tick the "Enable support for deprecated Algorithms" if relevant (i.e. did you require it for the connectivity test)





Now set what you want to happen with the file once its processed. If you want it to be deleted select that




or if you want it to be moved to an archive folder simply select move file and provide the sftp archive folder location




Finally select the schedule i.e. daily every 10 seconds etc. this is how often it will check to see if a new file has been created that matches the processing criteria








Step 6 - Modify file contents
This is probably the core learning of this process, once you have the file you can now use script (javascript of Groovy script) to essentially do anything to the contents of that file, before sending it to the target file.

To add a script entry choose the message transformers->script->Groovy Script







select where you want to drop the script




Select the create script icon





basic script will be injected into the new script. note the body value will contain the contents of the file





Step 9 - Get filename and pass to your script
Before we get to the step of processing the file and manipulating its contents I just want to cover how to use header parameters and access the name of the file that is passed. This example triggers based on a file name that starts with filename* so it could be filename-user123.csv, filename-20230101120000.txt etc etc but what if you wanted this date/time stamp or user id to use within your processing...maybe you want to create a new file that has this data within it, or simple wanted to store this value somewhere. Anyway its very basic stuff to capture this but isn't 100% obvious and took me longer than it should have when I was starting out.

Basically first add a content Modifier element (before the groovy script) to your integrtion flow




Then add a new "Message Header" entry with the following details:
Action: Create
Name: HEAD_FILENAME
Source Type: Expression
Source Value: ${file:name}
Data Type: java.lang.String






You can then reference this value via its name within your script.


Step 8 - Groovy script to modify file contents
Now to the groovy script. The below example basically recieves a .csv file and adds the timestamp from the filename as an extra csv column, then this new file contents is created as a new file at the ftp reciever destination.

def Message processData(Message message) {

//Body
def body = message.getBody();
def Name = message.getProperty('HEAD_FILENAME'); //filename will be storeed here
def headers = message.getHeaders();

// get header property
Name = headers.get("HEAD_FILENAME");

//Retrieve file contents in string format
def body2 = message.getBody(java.lang.String) as String;
def messageLog = messageLogFactory.getMessageLog(message);

//Split file at line break
def lines = body2.split("[\n\r]");

// Storee file contents in log
messageLog.addAttachmentAsString(Name, body2, "text/xml");

//Get timestamp from filename
def tstamp = Name.split("-");
tstamp[1] = tstamp[1].replaceAll(".csv", "");

// Prepare to store new entries
def newbody = lines[0] + ",DateTime"+"\n";//new column header

//loop at file entries/rows
for (int i = 1; i < lines.size(); i++) {
lines[i] = lines[i].replaceAll("[\n\r]", "");

if (! lines[i].isEmpty() ){ //check if row is not empty
if (newbody == null){
//build new file contents
newbody = lines[i] + ","+tstamp[1]+"\n";
}else{
//build new file contents
newbody = newbody + lines[i] + ","+tstamp[1]+"\n";
}
newbody = newbody.replaceAll(" ,", ",");
}
}

//build new file contents on log
messageLog.addAttachmentAsString("Records Added:", newbody, "text/xml");

//store new entries which will be appended to new FTP file
message.setBody(newbody);

return message;
}


Step 9 - Basic Groovy script to create helloworld in file
The above script shows just what is possible with this CPI and script...hopfully you can see that essentially anything is possible. But below is a more basic example which takes in any file and over writes it's contents with the words "Hello World" plus the file name.

def Message processData(Message message) {
//Body
def body = message.getBody();
def Name = message.getProperty('HEAD_FILENAME'); //filename will be storeed here
def headers = message.getHeaders();

// get header property
Name = headers.get("HEAD_FILENAME");

//Retrieve file contents in string format
def body2 = message.getBody(java.lang.String) as String;
def messageLog = messageLogFactory.getMessageLog(message);

//store original file contents in log
messageLog.addAttachmentAsString("Original body:", body2, "text/xml");


// Prepare to store new entries
def newbody;
newbody = 'Hello World ' + Name;


//store new file contents in log
messageLog.addAttachmentAsString("Hello world added:", newbody, "text/xml");

//store new entries which will be appended to new FTP file
message.setBody(newbody);

return message;
}


Step 10 - Set receiver SFTP connection
That last step is to set the receiver/destination of the integration flow. For this example we are just going it back to the same SFTP destination,uing the same details as for the sender.

so to do this select the connector option from the End point




and drag it to there receiver, select SFTP from the popup list




Enter the SFTP loging details and file/path details. Also remember to select the "Enable Support for deprecated Algorithums"




note you can make the file type what you want it to be (the helloworld example might make more sense to be .txt)




also note the processing options





Step 11 - Deploy your SFTP based SAP integration
Your integration diagram should now look like this within an sender SFTP as the start, cloud modifier, groovy script and a reciever SFTP end. It is now time to execute your integration, to do this simply save and then press the DEPLOY button





select yes




your integration has now been trigger for deployment, click OK




now return to the main view screen and all tile for the "Manage Integration Content"




within the list (use the filter/search to narrow it down) you should see your integration with the satus starting or started. Also note the Restart and Undeploy buttons as you might needs these if you want to stop your integration from running.




Step 12 - Test your SFTP based SAP integration
In-order to test the integration you simply need to drop a file into the ftp folder your sender is checking so lets create a test file like the following and ftp to the correct location




again return to the main view screen and select the messages tile of the "Monitor Message Processing"




keep refreshing until you see your artifact with status completed, select the Attachments tab and click on the entries




you will be take to a screen that shows the outputs you added within the groovy script like the original file contents




and the records added





and this is what they should look like with the simple hellow world example groovy script










note you can had both groovy scripts available and just connect or disconnect them as required





Step 13 - Additional SAP integration development options

note you can have multiple integration in the one artifact, simple choose process->integration process




and a new integration flow will be added




then just add the reciever and sender via the participant option





Step 14 - Split prosessing so you can do multipl actions from one sender trigger
So what if you wanted to create two outputs from one input? It isn't immediatly abvious how you do this, but it is however very easy to do once you know how. Simply using the multicast object (one input two outputs) so you can split the integration processing so you coulf for example ftp the same file to two locations, or ftp a file and send to external api.


To create a multicast object simply choose Routing icon->Multicast




choose parallel




and click on the integration procress window to add it




Now drag the linking arrows so that the multicast object has one input that goes to two outputs (i.e. in this example two groovy scripts)




In this example will will also send the second file to the same fp location so we need to create a second receiver





Step 15 - Copy end meessage event

When creating the END event for the second path you can actually copy the existing one




then paste it




You should then have a second end message




which you can link up to the second processing path and receiver




Note you can also create the End Message event manually




and then link it up





Step 16 - Copy SFTP configuration
You can also copy the actual SFTP link configuration in the same way by choosing the copy configuration option.




followed by the Paste configuratiomn option on the target SFTP link




select confirm





Step 16 - Future learning

The above example gives you a good introduction to SAP integration suit / CPI but the next things to look at are:

exception handling
transporting/copying to difference system (i.e. QAS/PROD)
externalizing your parameters(allows for configuration based on difference systems)