Overwrite tmnet DNS in router DIR-615

In the past, I have read into how the site was "blocked" in Malaysia where web surfers unable to load some websites. I don't know what is the fuss about it as it certainly create a lot of talks in the social websites. The term blocked would probably be at IP level where if the access by a machine to a address of IP a.b.c.d is unreachable, then that is a really blocked. But the funny things is, some ISPs in malaysia do not actually blocked the IPs but merely make some hostname unable to resolve into IP.

So in this article, I'm gonna show you how you can get back your freedom. There are many public DNS out there. Just google public DNS would give you a lot of list. In fact, google has public DNS too. When you choose the DNS, be sure it is public trust able and speed of your machine to the DNS server is matter when you want a quick web browser able to determine the IP address quickly.

With that said, I'm gonna show you how you can overwrite the DNS entries in tmnet router DIR-615. You need to ssh to your router in order to accomplish this. I have tried many times in the web interface, though it has text box for you to specify, apparently you cannot specify to the list that you want, it always default to tmnet DNS. This is annoying!

What you need is a terminal, ssh program installed, username = operator , password = h566UniFi

After you have ssh into the router, these are all the steps you need to do.

# cat /var/etc/resolv.conf
# This file is generated by gen_resolv.php
nameserver 1.9.1.9
nameserver 202.188.0.133

# sed 's/1.9.1.9/192.168.10.2/' -i /var/etc/resolv.conf
# sed 's/202.188.0.133/8.8.8.8/' -i /var/etc/resolv.conf
# cat resolv.conf
# This file is generated by gen_resolv.php
nameserver 192.168.10.2
nameserver 8.8.8.8

So I have an internal IP because there is a internal DNS service running. 8.8.8.8 would be the google public DNS.

Well, that's all you need, the site where it is unable to resolve to an IP, you can check by ping to the hostname, it should return a valid IP. That's all folks, I hope you learn something and gain back your right.

cassandra component on heap or off heap.

A summary of the components in cassandra that store offheap.

cassandra 1.2 and later use offheap for
bloom filter
compression offset map
row cache

cassandra 2.0 and later use offheap for
bloom filter
compression offset map
partition summary
index samples
row cache

Remember , off heap is not equal as non heap. Off-heap means the object is stored non in java heap allocated to the jvm, that is the native memory.

For example in the Bloom filter.

public static IAllocator newOffHeapAllocator(String offheap_allocator) throws ConfigurationException
{
if (!offheap_allocator.contains("."))
offheap_allocator = "org.apache.cassandra.io.util." + offheap_allocator;
return FBUtilities.construct(offheap_allocator, "off-heap allocator");
}

As you can see, if you want to determine if the object is stored on heap or offheap, you can to read into the code. I tried to inspect cassandra 2.0.2 using jconsole, it only show total usage of onheap or offheap. The cassandra community suggest it is better to read NEWS.txt or their blog to read the latest change if the component move on or off heap.

http://www.datastax.com/documentation/cassandra/2.0/webhelp/cassandra/operations/ops_tune_jvm_c.html
https://groups.google.com/forum/#!topic/nosql-databases/kFHS6QfZDuE
http://www.yourkit.com/docs/kb/sizes.jsp

force sstable compaction through jconsole

Happy new year everyone, this is my first article for 2014 and as a start, it is going to be short and sweet one. :-)

I was working on this project where I have deleted the column and I remember the definition of tombstone is that, you need to run compact so that the tombstone is removed. When I run nodetool compact, this message appear below.

INFO 19:15:09,170 Nothing to compact in index1.  Use forceUserDefinedCompaction if you wish to force compaction of single sstables (e.g. for tombstone collection)

So what is this means is that, you need to have jconsole running because forceUserDefinedCompaction can only invoke through jconsole. When jconsole is connected to your cassandra daemon process, you need to navigate to the compactionmanager mbean.

Then you need to provide two value to this method as this method expected two parameters. That is the keyspace and the sstables where you wish to compact. You can see in this attachment on how I did it.



So if you ls the data directory where the sstable are store, a new sstable should be generated. Once the operation is complete, you can navigate to the data directory where the sstable is stored.

$ ls data/lucene1/
index1-hc-3-Data.db index1-hc-3-Digest.sha1 index1-hc-3-Filter.db index1-hc-3-Index.db index1-hc-3-Statistics.db snapshots

We have production machine which a single sstable size is more than 25GB. When the soft limit compaction is met (default min 4 and max 32) , in this situation, I think it will load the server if it takes place, probably best is, we forceUserDefinedCompaction on this single large sstable.

Happy new year!

My software pick of 2013
* So the year is coming to the end, time to evaluate, when I look back at
2013, to see what software I used the most, compare to 2012, one piece of software
come to mind vimwiki, a personal wiki system for vim, I know it don't sound
like much, but it is just so much more, the wiki part refer to the way it use
links, in the previous version vimwiki used CamelBack to make a link, in the
latest all words can be used as a link, and you can link to files on your pc
or on the web.

from the homepage:http://code.google.com/p/vimwiki/
* With vimwiki you can
* organize notes and ideas
* manage todo-lists
* write documentation

* I use it for many purpose, note, blogs, todo, appointment, projects etc.
Before vimwiki I used zim it's a nice gui wiki, but the great think with
vimwiki is all the time I used to learn navigate in vim pays of, so I jump
around in my wiki-pages with a speed no gui tool can match, and all is kept
in text files, so the footprint on the harddrive is very small.

* I been using vimwiki for most of 2013 and my wiki-folder is only 392K, so
it's fast even if you put it on dropbox or local server, vimwiki has many
options to fit your needs, look up the vimwiki help file to learn more, a nice
thing is that you can have as many wiki's as you need, like one for work and
one private, one for a single project. I can't encourage you enough to read the
help file. So you can get the most out of this handy tool. I use it all day
every day, the speed and power from vim, in a wiki-tool, don't get better
then this.

* Thanks to habamax for this wonderful plugin.

* What is your software pick of 2013?

Cassandra cluster jmx metrics inspection to decide if cluster expansion is justifiable

One of the important decision during managing the clusters is to determine when cluster capacity should expand. To maintain a cluster in optimal performance will give the applications working nicely and most importantly, it give confidence to the people.

So, to answer question like, how do I determine if my cluster is at bottleneck? To answer this type of question, you will need to have the measuring tools ready and measure over time. Meaning that you need to display statistics in graphical form and with the history, it should give an indication of the cluster performance.

Because the topic will grow huge, hence, we will focus on a specific metric. This article gonna inspect the metric exposed by the jmx beans. In order to inspect the jmx metrics, you will need a jmx client. There is a gui jmx client that comes with the jdk, that is jconsole. Because nature of this article, I would suggest you go for cli jmx client, for example jmxterm. You can read introduction of jmxterm here.

There are many metrics exposed by cassandra jmx beans. But we will focus on bean org.apache.cassandra.db:type=CompactionManager.

If you are using jmxterm, you can read the output below:

$ cat test.script 
open localhost:7199
bean org.apache.cassandra.db:type=CompactionManager
get PendingTasks
$ java -jar jmxterm-1.0-alpha-4-uber.jar -i test.script 
Welcome to JMX terminal. Type "help" for available commands.
#Connection to localhost:7199 is opened
#bean is set to org.apache.cassandra.db:type=CompactionManager
#mbean = org.apache.cassandra.db:type=CompactionManager:
PendingTasks = 0;

So if you plot PendingTasks in a graph over time with a periodic interval, it should give insight to your cluster performance. You can also plot the statistics output from nodetool tpstats. I would suggest also, you plot Message type dropped as those metrics indicate over time that the performance is impacted. If you have a stock cassandra settings, you will probably want to fine tune to your node at this point after this investigation and analysis on the graph.

There is no best strategies, as mentioned earlier, you need experience and there are many other metrics measuring tools, example sar , iostats, top, your application measurement. So it take some times to even master all of these but it is crucial if you managed a production cluster.

cassandra 2.0 catch 101 – part5

Our cluster status.

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address         Load       Tokens  Owns   Host ID                               Rack
UN  192.168.33.31  147.34 KB  512     34.3%  f13c9390-4c52-4fc2-afa8-f7f74e7fd710  rack1
UN  192.168.33.32  123.31 KB  512     33.2%  bc7fcfcc-9a30-4929-bf24-35ec770856a3  rack1
UN  192.168.33.33  160.74 KB  256     16.5%  999d58bf-2b31-49ff-a452-6f0d01598429  rack1
UN  192.168.33.34  137.39 KB  256     16.0%  222796e9-d330-469a-8dcd-3f3581c9d795  rack1

So it is pretty interesting that a node can own different amount of cluster load based on the tokens specified. Because in our cluster environment, we have different types of hardware and for instance, mine is pretty old. With default settings, a default of 512 tokens is assigned.

The setting for tokens can be found in cassandra.yaml

# This defines the number of tokens randomly assigned to this node on the ring
# The more tokens, relative to other nodes, the larger the proportion of data
# that this node will store. You probably want all nodes to have the same number
# of tokens assuming they have equal hardware capability.
#
# If you leave this unspecified, Cassandra will use the default of 1 token for legacy compatibility,
# and will use the initial_token as described below.
#
# Specifying initial_token will override this setting.
#
# If you already have a cluster with 1 token per node, and wish to migrate to
# multiple tokens per node, see http://wiki.apache.org/cassandra/Operations
num_tokens: 512

Just note that this setting is one time only when a join the cluster. Meaning that the first time it join, 512 tokens will be assign for this node and this tokens will be store in the keyspace system and in column family local. Even if you removed the data directory and start all over, the data will be stream from other nodes, hence, the information is still persists. If you really want to change at later day, it is possible, you may want to treat this node as dead through decommission, stop cassandra instance. Change the num_tokens configuration in yaml file and then start cassandra instance back. You may want to think about this because decommission stream data to other servers and it may create load in system and also network traffic.

Lightweight Java Game Library

Since childhood, gaming has been one of my favorite activities. If you are from 80s, Supermario should sound familiar to you. =) 30 years had passed, gaming development improve tremendously over the period.

In this article, we are going to explore gaming development. Most of the gaming is written in low level languages, example C and thus, it is very complicated. This certainly introduced steep learning curve if you are a beginner. Hence, we will choose a simple startup to learn about gaming development. A example of library that can be use is Lightweight Java Game Library or its acronym LWJGL.

What is Lightweight Java Game Library?

The Lightweight Java Game Library (LWJGL) is a solution aimed directly at professional and amateur Java programmers alike to enable commercial quality games to be written in Java. LWJGL provides developers access to high performance crossplatform libraries such as OpenGL (Open Graphics Library), OpenCL (Open Computing Language) and OpenAL (Open Audio Library) allowing for state of the art 3D games and 3D sound. Additionally LWJGL provides access to controllers such as Gamepads, Steering wheel and Joysticks. All in a simple and straight forward API.

Because nature of this library deal with graphic display, hence the hardware display driver must be setup correctly. For me, my workstation is using ati radeon, and using xserver-xorg-video-radeon and enable 3D acceleration with package libgl1-mesa-dri. We won't delve deep into graphic driver installation and configuration since our focus here is the gaming development. You can check if your drive is setup properly by running glxgears via a terminal. If a windows popup with three gears spinning, your driver install and setup should be fine to continue for this coding tutorial.

In the official wiki, it is well written and documented to get you started. With this, I have setup my eclipse environment in debian sid. The library needed to should be setup in the project build path so when you run your application, the library is detected. Because I'm running linux, the native library location is pointed to lwjgl-2.9.1/native/linux. These two library must be configured before any development begin. If you noticed, I've setup the source as well, it will be convienient to read the code if you need to be sure later down the road during coding phase.



There are many tutorials to pick from, as a start, I just pick the basics - LWJGL Basics 1 (The Display).  The source code should be in the link, and it is incredibly easy to create the display with few lines of codes and I got that window display with just initial try. Very impressive and promising.



It is pretty impressive what this library can do. There are many examples that come in the library and one of it is an example game. Just execute

java -cp .:res:jar/lwjgl.jar:jar/lwjgl_test.jar:jar/lwjgl_util.jar:jar/jinput.jar: -Djava.library.path=native/linux org.lwjgl.examples.spaceinvaders.Game

if you are running linux. Run fine in my environment and played the bundle game; amazing. Maybe in my next article, I'm gonna try to even complete this .