Connecting to MongoDB service on bluemix using Node JS application and inserting data

In this blog,I will show you how to connect to Compose for MongoDB service on bluemix from our Node js application and then how to insert some data in to a collection.
First thing is that we need to create a new Compose for Mongo DB service On Bluemix.

mongodb.PNG

In the credentials you will have uri which is important and you need only that to connect from your node application

uri

Once you have  these you can start with your Node js application.
You require mongodb npm package to connect to this.So you need to import this module other than the default modules.
We need to use MongoClient constructor to connect to mongodb

var express = require('express');
var router = express.Router();
var bodyParser = require('body-parser');
var MongoClient = require('mongodb').MongoClient;
var collectionName = "";
var options = {
  mongos: {
    ssl: true,
    sslValidate: false,
  }
}
var mongo=process.env.VCAP_SERVICES;
var port = process.env.PORT || 3030;
router.use(bodyParser.json());
var conn_str = "";
var mongoDbConn;

As I told earlier ,we need mongo uri to connect.We can directly hardcode that in our code or else we can bind a custom provided service(cups) to your bluemix application and access that value using VCAP_SERVICES.Second option is the best one as dont need any hardcoding.
So here we need to bind that service to our application on bluemix. In above code mongo is directly accessing that service. Now we need to capture the uri value from that and connect to mongo using the MongoClient which is done below

if (mongo) {
  var env = JSON.parse(mongo);
  if (env['user-provided']) {
    mongo = env['user-provided'][0]['credentials'];
    console.log(mongo);
    if (mongo.uri) {
      conn_str = mongo.uri;

      MongoClient.connect(conn_str, options, function (err, db) {
        if (err) {
          res.send("Mongo DB with conn ${conn_str} has error ${err}");
        } else {
          mongoDbConn = db;
        }
      });

    } else {
      console.log("No mongo found");
    }
  } else {
    console.log('No mongo found');
  }
} else {
  console.log('No mongo found');
}

You can also directly hardcode your value for mongo.uri value if required.
So once you have your mongo connection,you don’t need to connect every time when you need to add some data or insert anything using rest call .You just need to use mongoDbConn .
In below example I will show you a POST call which will insert data in to a collection using mongoDbConn

router.post('/insertnewevent', function (req, res) {
  var eventdetails = req.body;
  var data = { "customer": "harish", "transactionID":"3405123","transdate":"11/06/17"};
  
  var collectionName = "logdetails";

  if (mongoDbConn) {
    // list messages
    mongoDbConn.collection(collectionName).insert(data, function (err, cursor) {
      if (err) {
        res.send('mongodb message list failed');
      } else {
        res.send("Inserted the event successfully and reference ID is " + data._id);
      }
    });
  }
});

So this way can easily connect to your MongoDB on bluemix service from your node js application.
Note:You can combine all the 3 code snippets in order and it will work.
If you face any issues in connecting you can comment here.
Hope it helps someone!

Advertisements

Creating new database and user in mongodb on bluemix using Mongo CLI on windows

One we have Compose for Mongodb service on bluemix we want to create different users and add the permission levels according to database.Creating users directly from bluemix GUI is not possible.We need to use Mongo CLI for this.
On MAC,it is quite easy but on Windows it is difficult and we need to install SSL enabled mongodb to connect to mongodb on bluemix.

The software can be downloaded from the following link

Once the software is installed , open CLI and go to folder where mongodb is installed.
Then to connect to mongodb we need to paste uri_cli which is the ssl uri which u will find in credentials

mongossl.PNG

Then you will be connected to mongodb service on bluemix

mongocoonected

Now to see the available databases ,

showdbs

To create a new database
newdb.png
To create a new user and assign a role to the user use the following command

db.createUser(
  {
    user: "UserName",
    pwd: "Password",
    roles: [
       
       { role: "readWrite", db: "testdb" }
    ]
  }
);

usercreate

This way we can easily create user and assign permissions.To list the users use command >show users

Hope it helps someone!

Setup and Commands for Uploading/Downloading files from Objectstorage in Windows using Swift CLI

On Mac it is quite simple to upload and download files from object storage in bulk ,but on windows it is quite difficult since we dont have any available documentation for this.
I use Windows 8 and once I need to do a backup of all the files from object storage container .There are around 5000 pdf files and there is no option where we can directly download all these in a single go from object storage.We need to select one by one.
So then I thought of using Swift CLI and after long struggle I succeeded on windows.So hope this helps someone.

First we need to have Swift CLI set up on your local machine.
For that we need to install Python,PIP,Swift client and keystone client.

First we need to set up virtualenv since on windows it doesnt support the openstack version 3 and Object storage is based on V3 .
But fortunately, python provides a way to install various version of command via “virtualenv” mechanism. It is possible to run various version of python in a virtual environment. And surely it worked for Bluemix Object Storage service too.

First we need to download python on your local machine and then install it and set the path

For window python installation you can refer this

https://github.com/BurntSushi/nfldb/wiki/Python-&-pip-Windows-installation

Then we need to install pip. Even for that you can refer the above link.

Then open cmd in the browse to the folder where you have pip installed and then u need to install virtualenv

>pip install virtualenv

>virtualenv virt

Then go to virt folder location

>cd virt

Capture1

Then install python swift client,python-keystoneclient and ndg-httpsclient module.

>pip install python-swiftclient

>pip  install python-keystoneclient

>pip install –upgrade ndg-httpsclient

Capture2

Capture3

Environment is set now and we can use the swift commands.

Firsy we need to  set the environment variables to access the object storage which you can get those from object storage service credentials

osmask

The values like <user-id>, <password> and <project-id> can be obtained from Bluemix Object Storage service UI. Once the details are correct u are authenticated and connected to object storage.

Then we can see the containers list using the following command

swift list

Capture6

Below are the Swift commands for downloading and uploading

Download a particular file from container

>swift download <container-name> <file-name>

 

Download all the files with in a container

>swift download <container-name> .

 

Uploading file in to a container

Go to the folder location where you have these files

>swift upload <container-name> <filename>

 

Uploading all the files in a folder to a container

>swift upload <container-name>

 

 

 

Uploading a file in to object storage container using node js application

In the earlier blog I’ve showed  how to generate authentication token for our object storage .In this blog I will show how to upload file in to object storage using that token.
So once we have our token we can use that to store files in the container.
In this blog I will create an api , using which we can upload a file in to object storage container.
For uploading any file through api , it is better that the file data should be passed to api as base64.Then it should be decoded and stored in object storage.

For uploading a file to objectstorage an api is provided which is,

API_ENDPOINT + ‘/’ + pdf_container_name + ‘/’ + pdfFileName
METHOD : PUT
headers: {


‘X-Auth-Token’: token,


‘Content-Type’: ”

}
Here API_ENDPOINT is mentioned in the previous blog

Here token is the one which we generated and Content-Type depends on the file type.

First we will create a function which would accept the base64 data and the name with which it should be stored in object storage(eg: document.pdf)

function storepdf(basedata, pdfName) {
    var filename = pdfName;
    var promise = new Promise(function(resolve, reject) {
        var pdf_container_name = "accountdocs"; //containername in objectstorage
        var bufferString = basedata; 
        var pdfFileName = filename;              
        getAuthToken().then(function(token) {
            getStringAsStream(bufferString).pipe(Base64.decode()).pipe(request({
                url: API_ENDPOINT + '/' + pdf_container_name + '/' + pdfFileName,
                method: 'put',
                headers: {
                    'X-Auth-Token': token,
                    'Content-Type': 'application/pdf'
                }
            }, function(error, response, body) {
                if (!error && response.statusCode == 201) {
                    return (pdfFileName);
                } else {
                    return (error);
                }
            }));
        });
    });
}

We are passing the base64 data as Stream and then we are decoding the base64 data and pipe() that data to our request.
Then in our main api we will call the above function as below :

app.post('/storepdf', function(req, res) {
    var data = req.body;
    var basedata = data.base64;
    var filename = data.filename;
    var fileUploadtime = (new Date).getTime();
    var file = filename.split(".")[0] + "_" + (new Date).getTime() + "." + filename.split(".")[1];
    storepdf(basedata, file);
    res.send(file);
});

Here we are appending timestamp to the filename ,storing the file in object storage and then we are sending that as response to this rest call.

Then we call use this api at our client side and server side and then upload your document.

Hope it helps someone.

Object storage:How to generate credentials and token using Node js application on bluemix

Object Storage

It is one most useful service if you are developing applications on bluemix .It provides unstructured cloud data storage. You can store and access your content using this service.

You can store your files such as pdf,txt,mp3 etc. Also you can connect to another apps and services.

How can we store files

First thing you need to have a bluemix account.Then login to your account and select your particular space.In data and analytics you can find object storage service and create it.Creating a service is nothing but an instance of that service getting created in your space.Once you create your service, create a container. This can be easily done from Manage tab.Below image shows it

 Creating a container

Once the container is created you can directly upload your files using the bluemix GUI.But in the real time we need our files to be uploaded through our application may be using some rest apis..

How can we use this service through application ?

IBM Object storage is based on SWIFT api . The service uses OpenStack Identity (Keystone) for authentication and can be accessed directly by using OpenStack Object Storage (Swift) API v1 calls.
So first we need some authentication keys to be generated for this service to use through our application.

Creating service keys for authentication:

Login to your bluemix account using cloud foundry using cf login

Then use the following command:

         cf create-service-key Objectstorage_servicename My-Service-Key

Now we will list this credentials since we need this by our application to authenticate to the object storage.

cf service-key My-Object-Storage My-Service-Key

Once we have these details we can focus on our application.Object storage is developed in such a away that if we want to use any api it requires an authentication token which is generated using the above values.

So the most important thing is generating a token.To generate this token we have an api for  region specific where we need to perform a post call with auth details of userid,password,project_id
Below is the api for UK region and for US region we need to replace lon with dal

  https://lon-identity.open.softlayer.com/v3/auth/tokens

Using the generated token we can upload or download any file from the container.
Generating token through post man or soap ui is simple.But we need to generate this using our node js application.Below code snippet shows you how we can generate that using our node js application.

Generating this token using NODE JS Application

// Create a config object
var config = {};
// Specify Openstack as the provider
config.provider = "openstack";
// Authentication url
config.authUrl = "https://lon-identity.open.softlayer.com";
config.region = "london";
// Use the service catalog
config.useServiceCatalog = true;
// true for applications running inside Bluemix, otherwise false
config.useInternal = true;
// projectId as provided in your Service Credentials
config.tenantId = "8fa4009784be465fa601ddc998a05ba0";
// userId as provided in your Service Credentials
config.userId = "b30caec05e054063b3ac52ec4844586c";
// username as provided in your Service Credentials
config.username = "admin_c62585ab7f8d4322654d09db3871c0195389040a";
// password as provided in your Service Credentials
config.password = "o()V]9=tz)~.d7Mu";
var API_ENDPOINT = ['https://lon.objectstorage.open.softlayer.com/v1/AUTH_', config.tenantId].join('');

config.auth = {
    forceUri: "https://lon-identity.open.softlayer.com/v3/auth/tokens",
    interfaceName: "public",
    "identity": {
        "methods": [
            "password"
        ],
        "password": {
            "user": {
                "id": "b30caec05e054063b3ac52ec4844586c", //userId
                "password": "o()V]9=tz)~.d7Mu" //userPassword
            }
        }
    },
    "scope": {
        "project": {
            "id": "8fa4009784be465fa601ddc998a05ba0"
        }
    }
};
//console.log("config: " + JSON.stringify(config));
var getAuthToken = function() {
    if (AUTH_TOKEN === '') {
        return new Promise(function(resolve, reject) {
            var authUrl = config.authUrl;
            var authInfo = {
                auth: {
                    identity: {
                        methods: ['password'],
                        password: {
                            user: {
                                id: config.userId,
                                password: config.password
                            }
                        }
                    },
                    scope: {
                        project: {
                            id: config.tenantId
                        }
                    }
                }
            }
            request({
                url: authUrl  + '/v3/auth/tokens',
                method: 'post',
                body: authInfo,
                json: true
            }, function(error, response, body) {
                if (error)		
                    reject(error);
                else {
                    resolve(response.headers['x-subject-token']);
                }
            });
       });
    } else {
        console.log("AUTH_TOKEN" + AUTH_TOKEN);
        return AUTH_TOKEN;
    }
}

Once the token is generated we can interact with the object storage through our node js application. In the next blog I will show how to use this token to upload file in to object storage container through our app.

Hope it helps someone!

Architecture of IBM BPM

There are number of functional components in IBPM Iam going to tell the basics of those components.

processcenter

 

Process Center:

It is the brain of BPM.We cannot work without a process center because that’s where development happens.The process center is called as a shared model.It means that,no matter what is being done with the over all solution,there is only one common repository and a single representation of that solution.Because of this, it is impossible to get two phases of the same solution out of synch with each other.

For example two people are working on same artifact at a time(eg:bpd).For one who works first will be editable and for the other it will be read only until the first person saves.Thus the data will always be in synch.

The process center is actually comprised of three components:

Process Center Server:

Enables you to run the processes as you build them.When your process is ready you can run that on test,production environments.Thus the process server will provide different run time environments where the developer can run the processes at different stages.

Process Center Repository: It is responsible for managing the solution’s artifacts.

Performance Data WareHouse:

It is responsible for collecting and managing the historical data of all the BPD’s and tasks that are run in the process server instances or the process center environment

What is BPM and the history of IBM BPM

BPM:

It will help your business to grow whatever it may be healthcare,finance,banking,insurance etc.,Everyone has some business process and they want management of that business process so that they can track it,make themselves aligned with the SLA’s and finally they can work efficiently on their business so that they can grow.

In technical terms,it can be defined as a field of operation management,that focuses on improving the corporate performance by managing and optimizing the company’s business process.

It will help to grow your business because you are going to do management of your business using the technology,what ever the technology it may be.

IBM BPM:It provides a platform where the business processes can be easily described,implemented,executed and monitored.

History of IBM BPM:

In 2005,IBM released a Business process management product called WPS (Websphere Process Server).It was designed to meet all the needs of a customer  from BPM perspective.It followed Service oriented architecture(SOA) paradigm.It means that customers would have business services and those would be aggregated together to build business solutions.

WPS came along with new open and business standard called Business Process Execution Language(BPEL).BPEL would solve all the business choreography needs.

(Choreography:There is no requirement that services need to be hierarchically composed.That means,two services may talk to each other.The protocol running between them is choreography)

This language can be visualized in a flow chart style diagram.The WPS provided IBM’s implementation of BPEL.So, with both SOA integration and BPEL IBM product,WPS came in to market.

But, unfortunately ,WPS could not capture 100% business market.WPS is more focusing on SOA principles. But competetors on more focussing on business users rather than technology users.It is more technical stuff and a business user could not sit and develop/capture their business process.To overcome this problem IBM came up with a new product called (IBM WebSphere Business Modeller) aimed at business users.But the problem is that WPS requiring migration from modeler to WPS for execution.

Acquisition of Lombardi:

A third party company called Lombardi called Teamworks which was a stiff competetor for  IBM WPS.

In 2010,IBM acquired Team Works and renamed as WebSphere Lombardi Edition(WLE) and the first release was in june,2010 i.e., WLE 7.1.In Dec,2010 WLE 7.2 version released.Although WLE replaced TeamWorks,we can still see the original name in few places.For example variable name starts with tw (eg:tw.local.text) .

WLE was successful but,customers were confused because IBM provides two products,i.e WPS and WLE.So to overcome this problem IBM came up with a single product,where they integrated the best parts of WLE and WPS in a single run time environment called IBM Business Process Manager(IBPM) and they released IBPM version 7.5.To maintain compatibility with the previous WPS and WLE customers,they kept many concepts of WPS and WLE by themselves.

Strength of IBPM:

Business processes are usually studied and captured by the business analysts and Bpm analysts.But they will have very less technical knowledge and they could not spend extensive time to translate business process in to automation design or implemetation.

But the biggest strength of IBPM is that with in few sessions anyone can capture the process in an intuitive diagram. After capturing the process,it can be play backed(run) to validate whether the process is fine or not.If we found any errors or misunderstandings we can easily make changes in real time in can be play backed again.This can be repeated until the correct process is captured.

The biggest differentiator of IBPM with other products is that it has interactive and real time iterative walk throuh capability.