MongoDB – Document structure and relations

Document structure:

MongoDB will represent relationships between documents with:

  • Embedded data – The whole document can be retrieved in a single query like this. The drawback is that if the embedded document keeps on growing too much in size, it can impact the read/write performance. The document max size is 16MB.
  • Manual References – similar with FK in RDBMS. It is the approach of designing normalized relationship.
  • Database References – a document references documents from many collections.

Below some examples with each type of representation applied to different relations.

One To One Relations:

One to Many relations:

Many to Many relations:

Note: It’s recommended to use multiple key indexes (compound index) to make queries against embedded documents efficient.

Benefits of Embedding:

  • Improvement read performance
  • One round trip to the database

Database References:

There are three fields in DBRefs:

  • $ref: This field specifies the collection of the referenced document
  • $id: This field specifies the _id field of the referenced document
  • $db: This is an optional field and contains name of the database in which the referenced document lies

Unless you have a compelling reason (if you need to reference documents from multiple collections) to use DBRefs, use manual references instead.

The custDetails DBRef field here specifies that the referenced customer document lies in customerName collection under customers database and has an id of 56868a99a1de4cd4b08a25ab.

The following code dynamically looks in the collection specified by $ref parameter (custDetails in our case) for a document with id as specified by $id parameter in DBRef.

 Additional links:
  1. https://docs.mongodb.org/manual/applications/data-models-relationships/

 

MongoDB – CRUD Operations

MongoDB is a cross-platform, document oriented database that provides, high performance, high availability, and easy scalability.

Quick Command reference:

CRUD operations

Creating documents:

MongoDB is able to provide few options to import documents into collection: insertOne, insert, update (upsert) and save.

output:

  • A boolean acknowledged as true if the operation ran with write concern or false if write concern was disabled.

If you want to add multiple documents at once, MongoDB provides insertMany with ordered (default – it will stop at first error) or unordered mode – even there are errors MongoDB will continue the insert with the next document.

output:

[code language=”javascript”][/code]

Update with upsert = true

If set to true, creates a new document when no document matches the query criteria. The default value is false, which does not insert a new document when no match is found.

You can use save command to quick update/insert a document into collection:

Reading documents:

Cursors:

cursor.objsLeftInBatch() returns the number of objects remaining in batch.

Logical operators:

Array match:

Update operators:

where optional options_document has any one or more of the following optional parameters:

Example:

See https://docs.mongodb.org/manual/reference/operator/update/

Delete:

Data aggregation:

Please follow this link with additional info about Data Aggregation.

Users management:

Reduce function:

and the output:

Pattern matching with $regex = equivalent to “like %” in RDBMS

Pattern matching without $regex:

Map-Reduce:

Map-Reduce is a data processing paradigm for condensing large volumes of data into useful aggregated results.

map-reduce

Source: mongodb

Additional links:1.MongoDB card reference.pdf

JavaScript – the most popular OS

No one writes JavaScript anymore. They write Jquery (Jake Weary)

 OS (traditional) Javascript (OS)
 Memory Management Garbage Collector
 Graphics Subsystems  HTML/Css, Canvas, Web GL, Audio, Video
 Storage Cookies, IndexedDb, File API
 Security  Sandbox, SSL, CORS
 Threading Web Workers
 Events DOM Events, Callbacks
 Network WebSockets, XHR, Offline, Realtime
API Geolocation, Sensors

Install latest nodejs on ubuntu

Note for myself: If you want to “dockerize” latest version of nodejs don’t forget below:

or

for v5.0

MongoDB 3.2 into xUbuntu 15.10

After I looked over the the steps from: https://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/  I wanted to start the service with:

Unfortunately I had below error:

Failed to start mongod.service: Unit mongod.service failed to load: No such file or directory.

This error occurred due to the problem with the new Ubuntu (15 and ahead).

Default init system is systemd which was Upstart previously. So you need to install Upstart, reboot your system and here you go, you can now run mongodb service.

  • Install Upstart

  • Reboot your system

sudo service mongod start mongod start/running, process 3371

As alternative to install mongodb:

Edit: 05.01.2016: Just follow below steps as an alternative to fix mongod.service:

Create a mongod.service into /lib/systemd/system/

Then run the commands:

My /etc/mongod.conf file looks like:

 

 

Zsh and themes into Linux Console

NodeJS Non-blocking architecture

JavaScript is a dynamic, object-oriented, and functional scripting language. One of the features that make it win over Java Applets in the browser scripting war decades ago, it was its lightness and non-blocking event loop.

Bocking means that when one line of code is executing the rest of it is locked waiting to finish. On the other hand, non-blocking gives to each line of code a shot and then through callbacks it can come back when an event happens. Programming languages that are blocking (Java, Ruby, Python, PHP, …) overcomes concurrency using multiple threads of execution while JavaScript handles it using non-blocking event loop in a single thread.

http://adrianmejia.com/images/threading_java.png

Image from strongloop.com

Some companies like Paypal moved from Java backend to NodeJS and reported a increased performance, lower average response times, and development speed gains. Similarly happens to Groupon that came from Java/Rails monoliths.

Samsung M2020 on xUbuntu

Open gnome-terminal via xUbuntu dash or via shortcut ctrl+alt+t and paste following commands:

  1. Download driver from Samsung site:

2. Unpack the driver:

3. Install driver, with sudo command you need to enter your root password:

4. Now, you should be able to to add your printer via Printers program. Add it as network printer or simply plug it in via USB and it should work.

 

 

Microservices antipatterns

Takeaways from microservice antipatterns:

  1. Overzeal­ous Services – don’t go straight into microservices, (fol­low­ing the hype per­haps?) rather than start­ing with the sim­plest thing possible. Instead, you should start by doing the sim­plest thing pos­si­ble, iden­tify the hotspots as they become appar­ent and then extract them into microservices. Start with monolith.
  2. Schemas Every­where – avoid having a shared data­base between dif­fer­ent ser­vices. This cre­ates tight cou­pling between the ser­vices and means you can’t deploy the microservices inde­pen­dently if it breaks the shared schema. Any schema updates will force you to update other microservices too. Instead, every ser­vice should have its own data­base, and when you want data from another ser­vice you go through its API rather than reach­ing into its data­base and help yourself. Use semantic versioning.
  3. Spiky Load between Services – You often get spiky traf­fic between ser­vices, and a com­mon solu­tion is to amor­tise the load by using queues between the services. Use rabbitmq.com, redis, etc.
  4. Hard­coded IPs and Ports – One solu­tion would be to use a dis­cov­ery ser­vice such as con­sul or etcd, and there’s also Netflix’s eureka. Another solu­tion is to use a cen­tralised router. Both solu­tions require reg­is­tra­tion and dereg­is­tra­tion, and both require high avail­abil­ity and scalability.
  5. Dog­piles – If one of your ser­vices is under load or mal­func­tion­ing, and all your other ser­vices keep retry­ing their failed calls, then the prob­lem would be com­pounded and mag­ni­fied by the addi­tional load from these retries. The solu­tion here is to have expo­nen­tial back­off and imple­ment the cir­cuit breaker pattern. Many libraries such as Netflix’s Hys­trix and Polly for .Net are available. You can even use the ser­vice dis­cov­ery layer to help prop­a­gate the mes­sage to all your ser­vices that a cir­cuit has been tripped (i.e. a ser­vice is struggling).
  6. Debug­ging Hell – Debug­ging is always a huge issue in micro-service archi­tec­tures. E.g., a nested ser­vice call fails and you can’t cor­re­late it back to a request com­ing from the user. One solu­tion is to use cor­re­la­tion IDs. When a request comes in, you assign the request with a cor­re­la­tion ID and pass it on to other ser­vices in the HTTP request header. Every ser­vice would do the same and pass the cor­re­la­tion ID it receives in the incom­ing HTTP header in any out-going requests. When­ever you log a mes­sage, be sure to include this cor­re­la­tion ID in the log line.
  7. Miss­ing Mock Servers – When you have a ser­vice that other teams depend on, each of these teams would have to mock and stub your ser­vice in order to test their own services. A good step in the right direc­tion is for you to own a mock ser­vice and pro­vide it to con­sumers.  You can take it a step fur­ther, by build­ing the mock ser­vice into the client so you don’t even have to main­tain a mock ser­vice any­more. Use Any­point Plat­form or Swagger.
  8. Fly­ing Blind – There are more than a hand­ful of tools avail­able in this space. There are com­mer­cial tools (some with free tiers) such as NewRelic and Stack­Driver (now inte­grated into Google AppEn­gin), AWS also offers Cloud­Watch as part of its ecosys­tem. In the open source space, Net­flix has been lead­ing the way with Hys­trix and  some­thing even more excit­ing.

Links: