Salesforce Bulk API Integration using Mule ESB

09.05.2014

Salesforce CRM has been widely used in organizations as part of managing their customer interactions. However, Salesforce with the Cloud Delivery model, has become difficult and expensive for the organizations to custom code their integration of Salesforce with their existing on-premise systems.

Many organizations have a need for this integration in a cost and time effective way to automate their business processes.  As a solution to this problem, WHISHWORKS has a way to integrate the company’s existing systems with Salesforce using a modern, lightweight and low cost Mule Enterprise Service Bus.

WHISHWORKS and Salesforce Bulk API

WHISHWORKS has extensive experience utilizing the Mule ESB Anypoint Salesforce Connector to connect directly with Salesforce APIs. This connector enables users, access to full Salesforce functionality with seamless Salesforce integration.

In a business scenario, where there were huge volumes of data to be migrated to Salesforce from the company’s multiple existing systems, WHISHWORKS has implemented an effective way of integrating with the Salesforce Bulk API.

Architecture Diagram

Bulk API

Mule ESB flows have been designed in a way that can be reused for both initial and operational loads. Transformation of data has also been provided in the process of data import to give a standardized and consolidated form of data in Salesforce.

How did we tune Salesforce Bulk API

Bearing in mind the different constraints Salesforce Bulk API has, WHISHWORKS has tuned the batches uploading to Salesforce in an effective manner enabling seamless business automation between Salesforce and the existing Database systems.  Here is how:

  • Threading Profile Settings: Salesforce allows a maximum of 25 concurrent threads at a time. To restrict the concurrent calls to not more than 25, threading profiles have been created at the flow and the VM endpoint level in which the Salesforce calls reside.
  • Salesforce Batch Tuning: Each Salesforce batch being created is tuned such that the data size of the batch does not exceed 10MB. Tuning parameters have been configured to change the number of records each batch holds depending on the size of the entity.
  • Time Delay between each Salesforce Call: Loading huge volumes of data to Salesforce in batches running concurrently can cause Salesforce to take longer time for processing the batches. To avoid this, a time delay has been provided between each concurrent call to avoid overloading Salesforce.
  • Parallel Garbage Collection: To utilise the JVM memory efficiently while importing the data, Parallel Garbage Collection has been used to clean the Java objects that are not anymore strongly referenced.

All this was done on Mule ESB!

Benefits to the Customer

The Salesforce Integration with the organization’s multiple systems has provided the following benefits to the customer:

  • This has enabled the customer to be able to streamline and fully automate their business processes.
  • Scalability: Integration through Mule ESB has enabled adaptation to any new SOA infrastructure that needs to be defined as part of the company’s changing infrastructure.
  • Speed of Integration: With an underlying platform that contains a single development environment and a reliable multi-tenant architecture, integration to Salesforce has been quickly and efficiently built.
  • This has provided them with an ability to integrate more systems and to aggregate data for a consistent and accurate overview of business.
  • Significant cost savings by using low cost Mule ESB Enterprise.

 

Author: Harika Guniganti is a Masters of Computer Science graduate having 4 years of experience as a Technical Specialist with WHISHWORKS. An experienced hand at integration and Mule ESB,  Harika loves her cooking, crafts and travelling.

If you would like to find out more about how APIs could help you make the most out of your current infrastructure while enabling you to open your digital horizons, do give us a call at +44 (0)203 475 7980 or email us at marketing@whishworks.com

Other useful links:

Integrate Mule ESB with SAP

Custom Connectors in MuleSoft

Integrating MuleSoft with ServiceNow Incident Management System

Recent Posts