Basics of Mongo DB and integration with Spring Boot Microservices!

If you have always wondered what is Mongo or NoSQL Databases and would like to experiment with it, yes you have come to the right place. So lets get on with it instead of the so called introduction/TOC/contents boilerplates!

What is Mongo?

Mongo is a NoSql DB in fact a document database which is distributed, scalable and easy to query with the ability to even do transactions! Obviously its loved by millions and is very easy to integrate with loads of languages and tools. But let’s get a level deeper shall we?

NoSQL

Basically they are schema less DB’s unlike traditional relational DB’s or RDBMS like Oracle and Sql Server or Postgres and MySql.Schema less means you don’t need a pre-defined structure or data modelling like the scripts you need when you setup tables in postgres .So does this mean you can just dump any json like structure , images or any resources in a noSQL DB? YES! Isn’t it simple and easy to develop?

Different kinds of NoSql Db’s

  • Document databases  – store data in documents like JSON /YAML/XML.Eg Mongo, Elastic Search,CouchDB,DocumentDB
  • Key-value databases – each record has a key and a value like a dictionary or hash table.Most of your caches work on this model.Eg Hazelcast, Redis, Memcache
  • Wide-column stores – data in stored in tables, rows, and dynamic columns.Mostly used in Big Data processing. Cassandra, Hbase, Google’s Bigtable
  • Graph databases – data in stored in nodes and edges.Popularly used in social media apps, when you have a person and their connections and many other related info .

Pro’s and Con’s of NoSQL DB’s

Its basically great for quick uploads and fast retrievals of data with the ability to horizontally scale.Can store different models without constraints. Cons are obviously data duplication as these are not normalized and are not meant for transaction based apps which need ACID compliance.

Mongo

Since we now know what kind of NoSQL DB Mongo is , you would know that docs or unstructured data(Eg your customer object can just be saved directly instead of ER modelling and entity -model mapping) can be stored and retrieved pretty fast using mongo.But in addition you should know that there is always the capability to do indexing to improved reads and aggregation like group by in RDBMS.

Cut to the chase!

There are several quick ways to get started. Popular among them is to use Atlas DB from here. They give you some basic storage space in their cloud offering under community license which is enough to play around with. Else you can also spin one off a docker image. Once you have your mongo url you can go ahead and connect it using either a IDE or CLI tools as well. Since you are beginning , i would suggest go with an IDE, popular ones include the Atlas but i liked NoSQLBooster for Mongo DB and Studio 3T, these are free with some limited features, but more than enough for day to day development.

Basics

How do you relate to the sql world when i venture out in Mongo? Well to start off with you need a Database and if you don’t create, Mongo will create one for you . And the tables in Mongo are called Collections and each collection has lots of documents under them which all have a unique identifier object called _id. All the individual properties of the document are called fields (corresponding more like a column in sql) . Thats pretty much everything you need to know.

eg if you need to access a certain DB called “testDB” just query

use testDB

Then you would want to create a document under a collection called testingMongoDocs

testDB.testingMongoDocs.insertOne( { hello: “World” } )

Congratulations you have created your first document!! But wait i forgot to create the collection .Dont worry Mongo has taken care of it for you!

Here are some basic queries which i think might help you, mostly CRUD.

//Fetch first 100 rows in desc order
db.sampleCollection.find({}).projection({}).sort({_id:-1}).limit(100)
//Projection helps in display and sort with 1 is asc and -1 is desc
//COUNT
db.sampleCollection.find().count()
//DISTINCT
db.sampleCollection.distinct("emp_role")
//DELETE
db.sampleCollection.remove({status: "Ex"})
// AND Condition
db.sampleCollection.find({$and:[{emp_id:'0986'},{status:'Active'}]})
//Update many
db.sampleCollection.updateMany({$and:[{emp_id:'1042'},{status:'Active'}]},{$set:{"status":"Ex"}})
//In command when you have many values
db.sampleCollection.find({$and: [{dept_id:"1041"},{proj_id:{$in: ["1233","1336","1346"]}},{is_active: true}]})

Since you are truly on your way to becoming a pro at Mongo now that you have the basics under belt, probably you would want a lil challenging query as well? how about aggregation?

Here is a sample

db.sampleCollection.aggregate({$match: { $and: [{role: "Mgr"},{is_active:true}]}},
    {$group: {_id: "$dept_id",
                count: {$sum: 1}}},
    {$match: { count: { $gt: 1 }}}
    )

Now that you are familiar with most of the queries you would now want to dabble with Spring boot and wire your code.

Spring Boot Integration

Now coming to development using Spring Boot , as always there is a Spring jpa will help you connect with the Mongo. You will need to add the following in you maven or gradle build file.Example given for gradle.

implementation ‘org.springframework.boot:spring-boot-starter-data-mongodb’

Once we have added the lib, we can add a MongoRepository to a api, which is basically a Hibernate implementation of Spring Data JPA.Something like this.

@Repository
public interface CartRepository extends MongoRepository<CartActivity, String> {

    CartActivity getCartActivitiesByKey(String key);

}

As you can see you can extend the MongoRepository in your repository layer/DAO and its takes the Document you want to return and the key field type for the Document. But where is the Document ?Well you have to create one in your project for example here is a sample.

@Document(collection="cart_products")
public class CartActivity {

    @Indexed
    @Field(value = "key")
    private String key;

    @Field(value = "activity_timestamp")
    private Instant activityTimestamp;

    @Field(value = "value")
    private String value;
}

There are a few terms you might now be aware.@Document annotation indicates that it is a Mongo Document and take a collection name as param. Also @Indexed here indicates that this is going to indexed, because we will most likely query based on this field , so to improve perf we can add @Indexed to fields that we would want to index. Finally each property on the object has to be tagged with @Field to indicate that they will need to be saved in the Document .

Now coming to the final piece of the puzzle , you would want to add the application yaml or properties file with all the config values which will help you connect to the MongoDB .A sample is here below.

spring:
  data:
    mongodb:
      uri: mongodb://user:password@mongo_instance:27017/DB_name?authSource=role
  jpa:
    show-sql: true

logging:
  level:
    org:
      springframework:
        data:
          mongodb:
            repository:
              query: debug

If you have replica sets for your MongoDB you will need to have a little different uri(check here). What i have provided above is a standalone instance. The relevant param i have mentioned will need to be replaced with relevant values. authSource is only when you want to set access control and restrict to the roles that one can access. And whats with all the logging and other jpa attrb? Well i have added this to the config , if you want to see the exact query that is performed behind all the magic that Spring JPA and Hibernate does behind the scenes.


Resources

https://www.mongodb.com/what-is-mongodb

https://docs.mongodb.com/manual/reference/

Good Practices for Pull Requests

A good PR guide

   Steps
  • Pull request for each feature is a minimum when completing a story for the same.
  • Assign a reviewer and setup some time with him , and give adequate time to review .
  • Allocate adequate time for reviewee as well to address the points raised by PR and fix them in the sprint stories.
  • Finally meet once again with reviewer to ‘resolve’ all the feedback given and get approval of the PR.
  • Ensure that the code works and is tested before the merge.
  • Once the PR is approved , validate the build and static code are successful, before merging and delete your branch to maintain repo hygiene.
  Best Practises
  • Keep PR’s small, so that the review is quick and feedback will not take much time.Else most of the time if you have a long living branch with lots of files,its very tedious to go throught various features and requirements to understand what was the expected behaviour.
  • Give a proper description for each PR, with changes implemented nicely listed.
  • Respect the time of both reviewer/reviewee, so that a proper time to fix a time with your reviewer. 
  • As a reviewer, it’s your responsibility not to hold on to the PR for too long , lest you end up blocking deployment and release.
  • Give constructive feedback for each review and not generic statements or mocking .Being rude is not an objective here, even if the code is frustrating to review.
  • The reviewer can ask the reviewee questions to understand more on the functionality or context.
  • Commented code or temp code should be avoided and maintain coding practices including testing and static analysis.
  • Always watch out for sensitive data not being checked into git.Eg config files with password.
  • We should always protect the main release branch from being merged directly without a PR process.

TIP: You now have the ability to review/start a PR within Intellij itself.

https://google.github.io/eng-practices/review/developer/small-cls.html

Git repository and branching standards

Guide for a new Repository

  1. Tag your Git repos along with application name or common theme which is relevant for that repo.Eg you might have several repos related to a proj called MyCart, tagging all the repos with MyCart will help discover them faster .
  2. Add readme with endpoints and support doc whenever possible.At the minimum each repo should atleast mention the core responsibilities of the repo.
  3. Enable Branch protection to avert inadvertent pushes.
  4. Readme updates in case of special needs like building the proj.
  5. Adding last Build run statuses using urls to you pipeline.

There is no limit on how far we would want to refine our git standards, but i have only concentrated on the min here which are the first 5 points above.

Code Check-in Guidelines

  • Before starting any work, take the updated code from git
  • Cut a branch name prefixed with feature Eg “feature-” from dev
    • this can be tagged to create a build name prefixed with “b” in drone (Example: b54-123e1222)
    • test your changes locally 
  • Once testing is done, create a Pull Request from your feature-* branch to dev
    • On merge, this should create a build name prefixed with “dev” in drone(Example: dev64-ff4e1233)
    • deploy these “dev*” builds in dev environment
    • test your changes in dev environment.
  • The last merge happens from dev to master via another Pull Request
    • this can create a build name prefixed with “p” in drone(Example: p34-22e6e882)
    • deploy these “p*” builds in prod environment
    • test your changes in prod env
  • Once the changes are deployed to prod, delete the “feature-*” branch

** In some scenarios if you keep a stage branch you will want to merge your changes from dev→ stage before you move to master. This will facilitate a prod like environment for UAT .

https://help.github.com/en/github/administering-a-repository/about-protected-branches

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-status-checks

Unable to configure device language!

Well this post will help you in getting Alexa app setup for iOS guys in India especially people who bought their echo in US. Amazon Echo got me excited as it had got some good reviews when it launched! But since i was in India when it did , I decided to get one from some folks in the US .But that turned out to be a bad idea as Amazon decided it shouldn’t be released in other countries yet and blocked out the Alexa app from the Apple Store in other regions other than US.So if you had to install one you would most likely have to install one using some hacks or would have to get a US payment card and register yourself in Apple store with that card info. Well for some reason I didn’t try the hacks and the workaround and waited patiently for a year!

Come next year in the fall of 2017 , Amazon decides to give Google Home a run for its money and launches Echo .But its on invitation basis only for now.And anyways i was only waiting for the Alexa to be installed. Once it did in last week of October in Apple Store, I installed it only to be frustrated again.This time “Unable to configure device language Error 15:1:26:0:2” during setup post installation of the app. Well I felt like hitting myself against the Echo device. But after a few days better sense prevailed as I got myself into experimental mood. I knew it had everything to do with the regional settings and so I changed the country settings in Amazon to US and then tried setting up the Alexa and it worked just fine. Now where do you find the country settings? Strangely i could only find it in amazon.com domain and not in Indian domain(ie .in) . It was under Manage your content and devices under settings tab. If you don’t know how to reach Manage your content and devices you will get it under you account page once you sign in.

Once you have changed the country to US and then sign into the Alexa app it will setup up perfectly! Once you have set it up , you can always go and change it back to your country in the website and within it app as well.But the initial setup is the only hiccup!

Swagger with ASP.NET WEB API !!

Have you ever wondered if there was an quick alternative to the Fiddler and the Advanced Rest Client /Postman for that matter.Something that would be pretty close to code that can be helpful in documentation and help you quickly get up to speed on testing your restful API’s? Well of late Swagger has been pretty popular because its open source and its ease of configuration.Well you can be up and running in 5 min.Swagger helps us to build pretty solid rest API documentation . Do you want to read on or do you want to get into the thick of it?

Step 1 :Let us just go ahead and select the most common ASP.Net MVC application and select WEB API as Project Template.Just hit F5 and you can see that by default the project has two Controllers Home and Values. We will concentrate on ValuesController as it seems to have some methods exposed.Well once you have the project up and running you will notice that the web project hosted via IISExpress on some port will have a API option on the top left hand on the home page.Clicking on it will lead to http://localhost:[portnumber]/Help which will provide some basic documentation on the webapi exposed.But you will soon see how much more convenient ,user friendly and testable Swagger is.

ASPHELP

Step 2 :Time to fire up Swagger from nuget packages.To help us set it up in ASP.NET there is a open source project called Swashbuckle .

Type in the following in Nuget Package Manager Console.

Install-Package swashbuckle

swashbuckle

Expanding the App_Start folder in your SolutionExplorer you can see the SwaggerConfig.cs magically inserted with some default options set.Basically ensure that you have at least the following configuration under SwaggerConfig.Register() method.

var thisAssembly = typeof(SwaggerConfig).Assembly;

GlobalConfiguration.Configuration
.EnableSwagger(c => c.SingleApiVersion(“v1”, “A title for your API”))
.EnableSwaggerUi();

Thats pretty much it .And you can run it and hit the below url to go to swagger url

http://localhost:%5Bportnumber%5D/swagger

swagger

Well you can now click on the Get url ,expand it and hit the try it button to see the request url,response body,response code as well as the response header.You can pass in the required parameters as you can discover using the second Get url.

You can also add xml documentation by linking the document to swagger in the SwaggerConfig.cs

First you will have to enable the xml documentation in Build tab under project properties .Next you will have to add the following line to EnableSwagger()

c.IncludeXmlComments(filePath:System.AppDomain.CurrentDomain.BaseDirectory + “[name of xml file mentioned under build tab]”);

And just restart the application.

swaggerxml

So you can see your xml comments reflected now.Cool isnt it?
You can even add Implementation Notes by adding remarks tag in your xml comments.You can also add HTML elements into the remarks tag if you are interested like adding a table inside the remarks tag.

To add response type schema you will have to decorate your web api action method with ResponseType tags as shown below.

responsetype

Just to add to the fun , you can add that extra user friendly documentation by adding a sample request json for your request uri.For example most of the POST action methods in an api, can be a little tricky if you dont set the mandatory properties properly.You will however need to implement a small interface called ISchemaFilter as follows.You will need to set your sample json to @default property.

public class CustomerDefault : ISchemaFilter
{
public void Apply(Schema schema, SchemaRegistry schemaRegistry, System.Type type)
{
schema.@default = new { Age = "34", Name = "John", Address = "PO Box 565,24th Down Street,Newcastle" };
}
}


Finally just decorate your class with the SwaggerSchemaFilter tag as shown below

[SwaggerSchemaFilter(typeof(AssignDPCIInfoDefault))]

Thats it you will get a sample json request under the webapi method, once you fire up your swagger ui.

There are loads of customization that you can do via swagger.I would encourage to visit the below url at your leisure

https://github.com/domaindrivendev/Swashbuckle

Top Apps for Apple Store

So here I come out of the dark ages of PC and Laptops and notebooks and here comes my first blog from a 4.7 inch smartphone . I wanted to share what apps I love most on iPhone and don’t worry all are free 😛.  So here are a few. Mind you these are my list ( from a Indian perspective)and you are free to have ur difference of opinion.

I am assuming most of us would have installed all the major social networking sites. You might also want to install about.Me for profiling and Flikr for cool snaps you have taken!!



They have a good interface and some great profiles to go through.

For taxi rides I use Uber and Olacabs (only in India)



The advantage with these mobile apps of regular taxis are they give you referrals and free rides which is not available when you their IVR or websites.merucabs are another local Indian fav.



Of news and feeds I rely on Flipboard and Instapaper and Digg.



( sorry but these are regular affair in India and I really wanted to support this brave lady)

Rely on Instapaper and Pocket for reading later when offline or while on travel where u don’t have internet.



Makemytrip ,Expedia , tripadvisor and cleartrip for travel bookings . Hotel right now from makemytrip gives good deals as well.



Coupondunia for deals and Zomato for review on places to have meals !



if you are a football fan like me you would like to install onefootball which gives you content of goal.com and you will get live ticker for games in progress as well from all the leagues in the globe. And if you love cricket then cricbuzz is the app to have.

I know for jobs most ppl use naukri and timesjobs, but glassdoor is invaluable to know the package and the culture of the company !



For rentals and searching houses you can use 99acres,common floor  , majicbricks



For pregnancy apps I can’t think of anything better than babycenter my pregnancy app.

And for reviews if most of  things I use mouthshut..

I use relax m for concentration and soothing sounds.

Ted for great talks and coursera for great computer talks . Meetups for scheduling and reaching out to like minded communities and groups mostly in the tech realm.



I use healthkart+ and webMD for meds and procedures and diagnosis.

Flipkart Ebooks gives you free Ebooks to read . Not just the romantic and the classics.

Justdial for any info on utility in neighborhood

Timeful for managing my time and yes finally WordPress and blogger apps for blogging 🙂

There are a few notable apps like memorado for brain games, busuu for learning foreign languages,topic to insert voice into snaps, awesome scanner for scanning docs and wireshare for sharing stuff . Try ipash tone kewl among geeks and I use procam for editing snaps when not using Instagram!!

There are a few other mentionable apps like my bsnl for paying your bsnl bills or mobile one for Bangalore utility bills and my Vodaphone for Vodaphone users finally 

Load Testing

I guarantee you this blog is going to be short and sweet but yeah it will add 2cents to your Info DB! So lets start ,how do you do load testing? Have used the load tests in Visual Studio? Its pretty much a breeze if you are familiar with the IDE.You can do a web performace test or a load test in Unit tests

Load1

And you can even step up the number of users(read traffic) or have a constant load.

load2

You can do web testing in the same manner as you normal methods in C#.

load3

In addition you can also try other tools like Fiddler.Simply pass the url that you wish to test in Composer section and repeat the call selecting the session and hitting Shift + R.You will get the results under statistics.Thats as simple as it can get.

load4

There is another amazing tool called stressstimulus which hooks with the fiddler and provides some cool features.
stresstimulus

You can see a quick demo here (http://forums.asp.net/t/1666267.aspx?Free+Load+Testing+Tool+Fiddler+add+on+StresStimulus)
Lastly there are now lots of free sites (http://loaduiweb.org/), where you can test your rest urls and load test them.But now sure if you are ready to test is on another box in case you are working for some corporate organisation.

Threats , Risks, Security – OWASP ?

For those who are newbie to security — and by security i dont mean authentication and authorization alone,this maybe an eye opener.Heard about CRSF and XSS- they are more in case you read on? Read this for an eye opener about some crazy stuff(http://www.troyhunt.com/2014/10/find-crazy-stuff-in-mobile-app.html).Troy has done some great stuff and sure you can check out his pluralsight video as well.But there is something that we all realise that we hardly give too much importance to security until the shit hits the fan.There have been more and more cases of hackers stealing credit cards from retail merchants and publishing them ,but we should all understand that he was able to do because “Developers were ignorant” .Not that i want to run away- I AM A DEV GUY too! But off late there is this new heat being generated that we are not giving adequate importance to these risks or vulnerabilities which can be exploited and these security threats can soon be a nightmare you dont want to take! Ok enuf said now what ? where do I start?
Simple — heard of OWASP ? You need to make sure that you pass these top 10 security risks. You can also go to owasp.org to learn more about it.Every year they publish the latest threats and for different platforms as well.The top 10 threats of mobility are
https://www.owasp.org/index.php/Mobile_Top_10_2014-M10

Ok now that i know what my vulnerabilities are ,how do i resolve it or how to i include it in SDL(software development lifecycle).First as a part of the process you can use Thread Modeling Tool from Microsoft(its free in case you thought $$$ just now 🙂 ) ,which is basically a modeling tool like visio but where you can see the vulnerable areas in design phase itself so that you are aware of your attack surface area and try eliminating vulnerable ones.You can download it from here .
http://www.microsoft.com/en-us/download/details.aspx?id=42518
Now to fix these issues in code you can either do google which by the way gives you lot more options or you can use a tool called Fortify from HP which plugs into you Visual Studio and gives you recommendations based on OWASP rules and severity and other filters as well.But sadly its paid and so are tools like Checkmarz and Klockwork. Ok so thats bad news, but sure enough i can guarantee you that if you have the patience google has the answers to most of the OWASP risks that your application has. So secure coding then 🙂

The terminal server has exceeded the maximum number of connections

Ever encountered this error and you have been frustrated that the last minute changes and hotfix is not going through.Well these things happen and more often than not we learn it at the wrong time 🙂 .There is an easy way to boot out unnecessary people from the server you are trying to login.Works wonders if you are a admin though.

To start off ,if you the nice guy and want people to be notified that you have a emergency and that you need to remotely login,then you need to know the users logged in .Some lousy admin or support guy may have forgotten to log off their sessions.Hence you need to find them and email them.Easier said than done.So sysInternals have for you a simple tool to help you find these guys.You can either download the entire suite from sysInternals website or they have a live version http://live.sysinternals.com/. Open PsLoggedon.exe and pass in your remote server name as shown below.

psloggedin

There you go,you now know who are the culprits.But still it might not resolve your problem and you might still not be able to login remotely.So when the push comes to shove,you have to bump someone out.Here are some commands to login forcefully .

To connect to a remote Windows Server 2003-based computer, go to Start and Run (Windows + R keyboard shortcut) and then type the following command:

mstsc -v:servername /F -console

NOTE: Newer versions of the RDP client require the following command instead.

mstsc /v:servername /F /admin

Happy Remoting!

Visual Studio Productivity Power Tools for VS 2012

Well for starters let me assume you are a poor dev like me and don’t have access to Resharper! Well in case you do they have a lot of these features that are now available free of cost in Productivity Tools in VisualStudioGallery.msdn.microsoft.com.

Let me quickly show you some of the features that stand out pretty instantaneously .

1.PresentOn/Off – type this into the quick launch and lo you get the power of zoomit 🙂

PresentOn

2.You can now edit a project file by right clicking on the project.

Edit Project

3.Browse to definition by pressing on control and left mouse click.

Definition

4.Email code snippets by using the same option in context menu.

Email

5.Now you can find duplicate code snippets by selecting the option “Find Matching clones in solution” in context menu.

Clones

Also there are other features like Auto Brace Completion,Open Command Prompt and HTML copy.
So go ahead and give it a download!!