Sunday, July 17, 2016

angelhack kuala lumpur 2016

Coding is something I have passion about and if you have been following me in my blog, you should know most of the article are big data technologies related. Two weeks ago as of this write up, a professor from MMU Malaysia asking me if I would like to join on his team on a hackathon event which held at the city centre in a mall, times square.

I took a look at the events which can be found here and here. To my astonishment it is a paid coding event which the currency operated at united state dollar even though we are in Malaysia. As this is something that I have never done before, that is, go to physical location and start to code for a project to compete against each other is one of my decisive factors and of cause, to know who are in the local development community, get to know more developers as well as have some fun!

There are mainly three challenges this year which are
* big data analytics
* o2o commerce
* smart living

and a grand prize which is an exclusive invite to a hackcelerator program and a chance to fly to silicon valley to compete further! Of cause, each categories come with prize pool!

Having practical background in big data technology and startup before, the professor think by inviting could form a team of five to compete on the big data analytics category. With that, I just give it a try and see how it goes. This event held on 4 and 5th June 2016 and of cause, on the weekend where just fit nicely with working professional schedule.

For my part, I have bring in big data technology and this time I selected elasticsearch as the nosql keystore which match with the challenge category we are aiming for. It was an unfortunate that since much of my time have been devoted to mid and backend development, my user interface development skill has been reduced tremendously. (Note to self, start to code GUI back!) But fortunately, to fix that gap, there was a good combination of software that came with elasticsearch, which is known as kibana fix what I have not bring to the table. Combination elasticsearch and kibana, we are not able to ingest the 65m of iproperty data into the machine, we are able to query and represent it nicely in colour graphs form.

As for the remaining of the team, which bring in skillsets like deep learning and predictive analysis which these two skillsets answer on 'predictive' and 'prescriptive' part. The idea of our product is that, user able to upload a picture of a property through a web interface (through a phone camera) and the system will learn that image and output the result using iproperty dataset that was store in elasticsearch and display in another page.

Below are some interesting pictures taken during the events.

the stage

panorama view

and the developers

As you can see from the above pictures, there are plenty of coders! Each skillful at what they are good at. There are estimated around 370 participants and that's including female coders too! Overall I think the organizer done really good job for this two days events. The participants actually got power, cool air, water, food, network devices and cable, junk food, tshirt, cup, writing notebook and the staffs are very helpful.

One of them actually came to me and asking what do I like, and actually handed me a paper cup with coffee as I requested. I could not express more gratitude since I got there around 8am and that's early for weekend. ;-) But I would like to point out there are definite areas to improve for next year.

* warm air on day one, it was not continuous but you feel the warm air.
* some of the schedule planned (the judging and present) written in paper is way too far off from the actual events flow.
* power and internet connection drop from time to time for two days.
* less junk food and perhaps replaced with fruits.
* the table arrangement too small for computer devices as well as personal comfortable.

But all in all, a good job to complete this two days event in a safe and sounds manner. If you know me, and the moto of this website striving for contribution and mutual benefits, I found out that there are still some and if not many of the developers who are very selfish in term of code sharing and/or code learning. For instance, all my codes and infrastructure are available online which can be found here.But in return, there are many excuses of them not share in between and something really go against my principle. I know that event such as this is compete against other team, I think what's more valuable above all these monetary values is the innovation and to cultivate coding passion. Something still I do not see this year and I don't believe we can thrive in closing environment and innovation definitely does not grow under such shallow person attitude.

hackathon ending in 5 minutes

pitching schedule

the team during pitching session.

Many coder actually slept on the mall but for me, home sweet home. I had a self reflection with the team and professor actually gave valuable insight on the night in slack and on the next days. Something that I have never experience since left academic. The coding session stop at 12pm on day 2 and then participant are free for lunch whilst the organizer and judges are preparing themselves for participant's pitching session. There were two rounds of pitching and each team just got about 2-3 minutes explanation their product and 1-2 minutes for question and answer.

It's a sad but reality that, during shortlisted team who pitch again on the stage, many judges keep using the word such as intellectual property, monetization, how you sell your products or how you make money out of your product. Honestly, these types of questions should be the responsible of the business segment professional. Event such as this should have been focus on the coding spirit and to actually value on the product that actually meet the set criterias. Yes, no doubt money is the topic of a company but that should have been an trivial issue for company, what's matter above is companies should be the main catalyst to drive the country to a better quality country. Not just project monetary society. Remember, the participant are as young as 15.

team effort

solo view

By the time the big data category announcement was made, I was dead tired.It's interesting to learn from other teams on the user interface design as well as the idea to transformed into the mockup prototype in just two days or less. To my surprise, we actually got the second prize for the challenge we compete for! That's worth 3000MYR and what's more valuable is a chance to be invited to Big App Challenge 3.0 Semi Finals. Hehe, whilst I fully acknowledge this reward rightfully belong to a team but a solo picture taken to acknowledge my contribution is not too much to ask for too.

Till then, keep coding and if you are interested to code, please feel free to contact me for any possibility to work together.




Saturday, July 16, 2016

Initial learning into apache cassandra paxos

Recently I have been reading into apache lightweight transaction in cassandra 2.0 and interested into how it implemented in code level. From end user perspective, when you manupulating data either insert and update with if not exists, then internally, paxos operation willl be used.

An example of lightweight transaction.

 INSERT INTO USERS (login, email, name, login_count) values ('jbellis', 'jbellis@datastax.com', 'Jonathan Ellis', 1) IF NOT EXISTS  
   
 UPDATE users SET reset_token = null, password = ‘newpassword’ WHERE login = ‘jbellis’ IF reset_token = ‘some-generated-reset-token’  

Essentially the paxos how operation concerntrated in class StorageProxy.We read that from the code documentation,

There are three phases to Paxos:
1. Prepare: the coordinator generates a ballot (timeUUID in our case) and asks replicas to (a) promise
   not to accept updates from older ballots and (b) tell us about the most recent update it has already
   accepted.
2. Accept: if a majority of replicas reply, the coordinator asks replicas to accept the value of the
   highest proposal ballot it heard about, or a new value if no in-progress proposals were reported.
3. Commit (Learn): if a majority of replicas acknowledge the accept request, we can commit the new
   value.

So it involve a few operation before an insert and update can be perform and this is not something you want to replace in bulk operation call with. We see that in class StorageService, several paxos verbs are registered. You can find them mostly in the paxos packages. Following are some useful paxos classes.


Interesting in the class PaxosState, noticed the following
locks - an array of length 1024
call system keyspace class to load and save paxos state.

When you check into cassandra using cqlsh, you will find the following.

 cqlsh:jw_schema1> use system;  
 cqlsh:system> desc tables;  
   
 available_ranges     peers        paxos      range_xfers  
 batches          compaction_history batchlog    local     
 "IndexInfo"        sstable_activity  size_estimates hints     
 views_builds_in_progress peer_events     built_views    
   
 cqlsh:system> select * from paxos;  
   
  row_key | cf_id | in_progress_ballot | most_recent_commit | most_recent_commit_at | most_recent_commit_version | proposal | proposal_ballot | proposal_version  
 ---------+-------+--------------------+--------------------+-----------------------+----------------------------+----------+-----------------+------------------  
   
 (0 rows)  

You can also read the unit test for paxos in cassandra 2.0.17 as can be read here.

This compare and set (cas) operation looks interesting if you want to ensure the value only be created if it not exists, a nifty feature found in apache cassandra 2.0 onward. Feel free to explore further!

Friday, July 15, 2016

learning and trying on apache ivy

Over years of java programming using ant build file, many have claimed that ant is an ancient automated software build process and you should instead use maven2. Why? Because they claimed that ant does not automate to resolve the libraries that you required in project. So instead of switching to use maven2 entirely, today we will take a look at apache ivy.

Apache Ivy is a transitive dependency manager. It is a sub-project of the Apache Ant project, with which Ivy works to resolve project dependencies. An external XML file defines project dependencies and lists the resources necessary to build a project. Ivy then resolves and downloads resources from an artifact repository: either a private repository or one publicly available on the Internet.
That is really a good news! Instead of starting all over using maven, now the libraries dependency can be solve via ivy in your existing ant build file. What a relieve! Okay, let's go into the basic of apache ivy. Let's start with the terminology that is use widely when you deal with apache ivy. You can reference this link for more details explanation. For concept of how apache ivy works, read this helpful link.

terminology explanation
Organisation An organisation is either a company, an individual, or simply any group of people that produces software.
Module A module is a self-contained, reusable unit of software that, as a whole unit, follows a revision control scheme.
Module Descriptor A module descriptor is a generic way of identifying what describes a module: the identifier (organisation, module name, branch and revision), the published artifacts, possible configurations and their dependencies.
Artifact An artifact is a single file ready for delivery with the publication of a module revision, as a product of development.
Type of an artifact The artifact type is a category of a particular kind of artifact specimen.
Artifact file name extension In some cases the artifact type already implies its file name extension, but not always.
Module Revision A unique revision number or version name is assigned to each delivered unique state of a module.
Branch A branch corresponds to the standard meaning of a branch (or sometimes stream) in source control management tools.
Status of a revision A module's status indicates how stable a module revision can be considered.
Configurations of a module A module configuration is a way to use or construct a module.
Ivy Settings Ivy settings files are xml files used to configure ivy to indicate where the modules can be found and how.
Repository What is called a repository in Ivy is a distribution site location where Ivy is able to find your required modules' artifacts and descriptors (i.e. Ivy files in most cases).



when you add the ivy namespace into your ant build file, you can call ivy task in ant build file. An example of the ivy integration into ant build file as show below.


 <project xmlns:ivy="antlib:org.apache.ivy.ant" name="go-ivy" default="go">  
 <!--  
    
     this build file is a self contained project: it doesn't require anything else   
     that ant 1.6.2 or greater and java 1.4 or greater properly installed.  
       
     It is used to showcase how easy and straightforward it can be to use Ivy.  
       
     This is not an example of the best pratice to use in a project, especially  
     for the java source code "generation" :-) (see generate-src target)  
       
     To run copy this file in an empty directory, open a shell or a command window  
     in this directory and run "ant". It will download ivy and then use it to resolve   
     the dependency of the class which is itself "contained" in this build script.  
       
     After a successful build run "ant" again and you will see the build will be  
     much faster.  
       
     More information can be found at http://ant.apache.org/ivy/  
       
 -->  
 <!--  
  here is the version of ivy we will use. change this property to try a newer   
      version if you want   
 -->  
 <property name="ivy.install.version" value="2.4.0"/>  
 <property name="ivy.jar.dir" value="${basedir}/ivy"/>  
 <property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar"/>  
 <property name="build.dir" value="build"/>  
 <property name="src.dir" value="src"/>  
 <target name="download-ivy" unless="skip.download">  
 <mkdir dir="${ivy.jar.dir}"/>  
 <!--  
  download Ivy from web site so that it can be used even without any special installation   
 -->  
 <echo message="installing ivy..."/>  
 <get src="https://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar" dest="${ivy.jar.file}" usetimestamp="true"/>  
 </target>  
 <!--  
  =================================   
      target: install-ivy       
       this target is not necessary if you put ivy.jar in your ant lib directory  
       if you already have ivy in your ant lib, you can simply remove this  
       target and the dependency the 'go' target has on it  
      =================================   
 -->  
 <target name="install-ivy" depends="download-ivy" description="--> install ivy">  
 <!--  
  try to load ivy here from local ivy dir, in case the user has not already dropped  
         it into ant's lib dir (note that the latter copy will always take precedence).  
         We will not fail as long as local lib dir exists (it may be empty) and  
         ivy is in at least one of ant's lib dir or the local lib dir.   
 -->  
 <path id="ivy.lib.path">  
 <fileset dir="${ivy.jar.dir}" includes="*.jar"/>  
 </path>  
 <taskdef resource="org/apache/ivy/ant/antlib.xml" uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>  
 </target>  
 <!--  
  =================================   
      target: go  
           Go ivy, go!  
      =================================   
 -->  
 <target name="go" depends="install-ivy, generate-src" description="--> resolve dependencies, compile and run the project">  
 <echo message="using ivy to resolve commons-lang 2.1..."/>  
 <!--  
  here comes the magic line: asks ivy to resolve a dependency on   
        commons-lang 2.1 and to build an ant path with it from its cache   
 -->  
 <ivy:cachepath organisation="commons-lang" module="commons-lang" revision="2.1" pathid="lib.path.id" inline="true"/>  
 <echo message="compiling..."/>  
 <mkdir dir="${build.dir}"/>  
 <javac srcdir="${src.dir}" destdir="${build.dir}" classpathref="lib.path.id"/>  
 <echo>  
 We are now ready to execute our simple program with its dependency on commons-lang. Let's go!  
 </echo>  
 <java classname="example.Hello">  
 <classpath>  
 <path refid="lib.path.id"/>  
 <path location="${build.dir}"/>  
 </classpath>  
 </java>  
 </target>  
 <!--  
  =================================   
      target: generate-src  
       'Generates' the class source. It actually just echo a simple java   
       source code to a file. In real life this file would already be  
       present on your file system, and this target wouldn't be necessary.  
      =================================   
 -->  
 <target name="generate-src">  
 <mkdir dir="${src.dir}/example"/>  
 <echo file="${src.dir}/example/Hello.java">  
 package example; import org.apache.commons.lang.WordUtils; public class Hello { public static void main(String[] args) { String message = "hello ivy !"; System.out.println("standard message : " + message); System.out.println("capitalized by " + WordUtils.class.getName() + " : " + WordUtils.capitalizeFully(message)); } }  
 </echo>  
 </target>  
 <!--  
  =================================   
      target: clean         
      =================================   
 -->  
 <target name="clean" description="--> clean the project">  
 <delete includeemptydirs="true" quiet="true">  
 <fileset dir="${src.dir}"/>  
 <fileset dir="${build.dir}"/>  
 </delete>  
 </target>  
 <!--  
  =================================   
      target: clean-ivy         
      =================================   
 -->  
 <target name="clean-ivy" description="--> clean the ivy installation">  
 <delete dir="${ivy.jar.dir}"/>  
 </target>  
 <!--  
  =================================   
      target: clean-cache         
      =================================   
 -->  
 <target name="clean-cache" depends="install-ivy" description="--> clean the ivy cache">  
 <ivy:cleancache/>  
 </target>  
 </project>  

As the saying goes, try it and you will understand better how apache ivy help you in your ant build process. Once you get a hang of it and you want more advance feature, I suggest you take a look at ivy setting file. This link provide comprehensive coverage of the configuration that you can use in advance use cases.

If you still look for what else you can do with apache ivy, take a look at this link. If you just want to quickly use ivy, you can use ivy as a standalone jar file.

I have not mentioned ivy.xml and i think if you have reach this section, you should know what's ivy.xml file for and what does it contain. I hope you found something useful in this quick tutorial.

Sunday, July 3, 2016

apache cassandra 1.0.8 on READ_STAGE threads reference on sstables and so compaction cannot remove the sstables.

Back then when I was administer a apache cassandra 1.0.8 cluster, I noticed there were some (very little) sstables did not get remove even after compaction is done. The leftover sstables cause some administrative problem and I suspect could be due to maybe during reading of the sstables, this maybe not get remove.

 DataTracker.java  
   
   private void replace(Collection<SSTableReader> oldSSTables, Iterable<SSTableReader> replacements)  
   {  
     View currentView, newView;  
     do  
     {  
       currentView = view.get();  
       newView = currentView.replace(oldSSTables, replacements);  
     }  
     while (!view.compareAndSet(currentView, newView));  
   
     addNewSSTablesSize(replacements);  
     removeOldSSTablesSize(oldSSTables);  
   
     cfstore.updateCacheSizes();  
   }  

I supposed during replacement of the view and sstables, everything is atomic and hence during read, it will get from the new sstables. But I don't have enough high level knowledge on various subsystems work in cassandra. If you have an idea, please do leave your comment below.

This problem seem to go away after we upgraded the cluster to 1.1. I know by now (april 2016), cassandra 1.0, 1.1 or even 1.2 is ancient but if you are on 1.0 and pre1.0, you should really start to use cassandra 3.x or at least 2.x.

Saturday, July 2, 2016

quick maven note for myself

If you are java developer and maven should be very familiar. So I was surprise I never write a blog about maven but now I might as well write a short one. A note to keep myself reminded and can be revisited later.

To start a new maven java project, specify groupId, artifactId and archetypeArtifacId.
 start a mvn project  
 $ mvn archetype:generate -DgroupId=co.weetech -DartifactId=my-webapp -DarchetypeArtifacId=maven-archetype-quickstart -DinteractiveMode=false  

Once you have the directory build, start to add code and then when it is time to compile, let's do it!

 compile the project  
 $ mvn compile  

Sometime you have unit test and you write your unit test in the path ./src/test and then you can run them together using this command.

 test the project  
 $ mvn test

and if everything is compile and tested properly , you can start package them. Well, maven does that as well.

 package the project  
 $ mvn package  

the built library should be in the target directory. Finally if you want to install the package in your local directory, under home .m2, you can run the following command.

 to install the package, into your home .m2 directory  
 $ mvn install  

and if your project grows bigger and you want to edit your source code in eclipse, run the following command to generate the file descriptors.

 maven eclipse  
 $ mvn eclipse:eclipse  

That's it, these should get you started and if you encountered problem, google is your friend :-D and of cause maven documentation.

Friday, July 1, 2016

Yet another sstable corruption - EOFException

During starting up a apache cassandra 1.2 instance, I noticed in the log of the following error.

 INFO 10:38:23,334 Opening /var/lib/cassandra/data/MYKEYSPACE/MYCOLUMNFAMILY/MYKEYSPACE-COLUMNFAMILY-hf-2508 (2275767 bytes)  
 ERROR 10:38:23,467 Exception in thread Thread[SSTableBatchOpen:2,5,RMI Runtime]  
 org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException  
    at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:108)  
    at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:63)  
    at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:42)  
    at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:418)  
    at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:209)  
    at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:157)  
    at org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:273)  
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
    at java.lang.Thread.run(Thread.java:745)  
 Caused by: java.io.EOFException  
    at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340)  
    at java.io.DataInputStream.readUTF(DataInputStream.java:589)  
    at java.io.DataInputStream.readUTF(DataInputStream.java:564)  
    at org.apache.cassandra.io.compress.CompressionMetadata<init>(CompressionMetadata.java:83)  
     ... 11 more  

Yes, if you noticed that the cassandra sstable version is hf which belong to cassandra 1.1 as this node is just right after cassandra 1.2 upgrade and first cassandra 1.2 boot up.

Tracing the stacktrace above with cassandra 1.2 source code, it turn out to be the compression metadata cannot be open due to file corruption. I tried using nodetools upgradesstables, scrub and restart cassandra instance, this error still persist. I guess in this case, nothing can really help so I end up stopping the cassandra instance. remove this data sstables together with its metadata sstables and then start it up again. The error is gone and I ran a repair.

I hope you find this useful in your situation too.

Sunday, June 19, 2016

Investigating into apache cassandra 1.2 jmx metrics connection type warn logging

Recently I got the opportunity to upgrade a production cassandra cluster from 1.1.12 to 1.2.19 and during the midst of upgrading, I noticed the following in the log file during boot up of a cassandra 1.2 instance.

1:  WARN 10:30:14,987 Error processing org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=Timeouts  
2:  javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=Timeouts  
3:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)  
4:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)  
5:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)  
6:     at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)  
7:     at com.yammer.metrics.reporting.JmxReporter.registerBean(JmxReporter.java:462)  
8:     at com.yammer.metrics.reporting.JmxReporter.processMeter(JmxReporter.java:412)  
9:     at com.yammer.metrics.reporting.JmxReporter.processMeter(JmxReporter.java:16)  
10:     at com.yammer.metrics.core.Meter.processWith(Meter.java:131)  
11:     at com.yammer.metrics.reporting.JmxReporter.onMetricAdded(JmxReporter.java:395)  
12:     at com.yammer.metrics.core.MetricsRegistry.notifyMetricAdded(MetricsRegistry.java:516)  
13:     at com.yammer.metrics.core.MetricsRegistry.getOrAdd(MetricsRegistry.java:491)  
14:     at com.yammer.metrics.core.MetricsRegistry.newMeter(MetricsRegistry.java:240)  
15:     at com.yammer.metrics.Metrics.newMeter(Metrics.java:245)  
16:     at org.apache.cassandra.metrics.ConnectionMetrics.<init>(ConnectionMetrics.java:106)  
17:     at org.apache.cassandra.net.OutboundTcpConnectionPool.<init>(OutboundTcpConnectionPool.java:53)  
18:     at org.apache.cassandra.net.MessagingService.getConnectionPool(MessagingService.java:493)  
19:     at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:507)  
20:     at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:640)  
21:     at org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:614)  
22:     at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:59)  
23:     at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)  
24:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
25:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
26:     at java.lang.Thread.run(Thread.java:745)  

As the logging level is WARN, I did not worry that much. Going into the codes, it turn out that in cassandra 1.2 , a metric known as ConnectionMetrics is added. This metric is under domain org.apache.cassandra.metrics and of type connection and name is Timeouts. This is not available in cassandra 1.1.

The same situation is applicable to CommandPendingTasks, ResponseCompletedTasks, ResponsePendingTasks, CommandCompletedTasks.


1:  WARN 10:38:58,079 Error processing org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=CommandPendingTasks  
2:  javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=CommandPendingTasks  
3:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)  
4:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)  
5:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)  
6:     at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)  
7:     at com.yammer.metrics.reporting.JmxReporter.registerBean(JmxReporter.java:462)  
8:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:438)  
9:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:16)  
10:     at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28)  
11:     at com.yammer.metrics.reporting.JmxReporter.onMetricAdded(JmxReporter.java:395)  
12:     at com.yammer.metrics.core.MetricsRegistry.notifyMetricAdded(MetricsRegistry.java:516)  
13:     at com.yammer.metrics.core.MetricsRegistry.getOrAdd(MetricsRegistry.java:491)  
14:     at com.yammer.metrics.core.MetricsRegistry.newGauge(MetricsRegistry.java:79)  
15:     at com.yammer.metrics.Metrics.newGauge(Metrics.java:70)  
16:     at org.apache.cassandra.metrics.ConnectionMetrics.<init>(ConnectionMetrics.java:71)  
17:     at org.apache.cassandra.net.OutboundTcpConnectionPool.<init>(OutboundTcpConnectionPool.java:53)  
18:     at org.apache.cassandra.net.MessagingService.getConnectionPool(MessagingService.java:493)  
19:     at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:507)  
20:     at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:640)  
21:     at org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:614)  
22:     at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:59)  
23:     at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)  
24:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
25:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
26:     at java.lang.Thread.run(Thread.java:745)  
27:       
28:   WARN 07:52:19,882 Error processing org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=ResponseCompletedTasks  
29:  javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=ResponseCompletedTasks  
30:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)  
31:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)  
32:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)  
33:     at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)  
34:     at com.yammer.metrics.reporting.JmxReporter.registerBean(JmxReporter.java:462)  
35:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:438)  
36:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:16)  
37:     at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28)  
38:     at com.yammer.metrics.reporting.JmxReporter.onMetricAdded(JmxReporter.java:395)  
39:     at com.yammer.metrics.core.MetricsRegistry.notifyMetricAdded(MetricsRegistry.java:516)  
40:     at com.yammer.metrics.core.MetricsRegistry.getOrAdd(MetricsRegistry.java:491)  
41:     at com.yammer.metrics.core.MetricsRegistry.newGauge(MetricsRegistry.java:79)  
42:     at com.yammer.metrics.Metrics.newGauge(Metrics.java:70)  
43:     at org.apache.cassandra.metrics.ConnectionMetrics.<init>(ConnectionMetrics.java:99)  
44:     at org.apache.cassandra.net.OutboundTcpConnectionPool.<init>(OutboundTcpConnectionPool.java:53)  
45:     at org.apache.cassandra.net.MessagingService.getConnectionPool(MessagingService.java:493)  
46:     at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:507)  
47:     at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:640)  
48:     at org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:614)  
49:     at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:59)  
50:     at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)  
51:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
52:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
53:     at java.lang.Thread.run(Thread.java:745)  
54:       
55:   WARN 09:06:07,059 Error processing org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=ResponsePendingTasks  
56:  javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=ResponsePendingTasks  
57:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)  
58:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)  
59:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)  
60:     at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)  
61:     at com.yammer.metrics.reporting.JmxReporter.registerBean(JmxReporter.java:462)  
62:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:438)  
63:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:16)  
64:     at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28)  
65:     at com.yammer.metrics.reporting.JmxReporter.onMetricAdded(JmxReporter.java:395)  
66:     at com.yammer.metrics.core.MetricsRegistry.notifyMetricAdded(MetricsRegistry.java:516)  
67:     at com.yammer.metrics.core.MetricsRegistry.getOrAdd(MetricsRegistry.java:491)  
68:     at com.yammer.metrics.core.MetricsRegistry.newGauge(MetricsRegistry.java:79)  
69:     at com.yammer.metrics.Metrics.newGauge(Metrics.java:70)  
70:     at org.apache.cassandra.metrics.ConnectionMetrics.<init>(ConnectionMetrics.java:92)  
71:     at org.apache.cassandra.net.OutboundTcpConnectionPool.<init>(OutboundTcpConnectionPool.java:53)  
72:     at org.apache.cassandra.net.MessagingService.getConnectionPool(MessagingService.java:493)  
73:     at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:507)  
74:     at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:640)  
75:     at org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:614)  
76:     at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:59)  
77:     at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)  
78:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
79:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
80:     at java.lang.Thread.run(Thread.java:745)     
81:    
82:    
83:   WARN 02:13:09,861 Error processing org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=CommandCompletedTasks  
84:  javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=Connection,scope=1.2.3.4,name=CommandCompletedTasks  
85:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)  
86:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)  
87:     at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)  
88:     at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)  
89:     at com.yammer.metrics.reporting.JmxReporter.registerBean(JmxReporter.java:462)  
90:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:438)  
91:     at com.yammer.metrics.reporting.JmxReporter.processGauge(JmxReporter.java:16)  
92:     at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28)  
93:     at com.yammer.metrics.reporting.JmxReporter.onMetricAdded(JmxReporter.java:395)  
94:     at com.yammer.metrics.core.MetricsRegistry.notifyMetricAdded(MetricsRegistry.java:516)  
95:     at com.yammer.metrics.core.MetricsRegistry.getOrAdd(MetricsRegistry.java:491)  
96:     at com.yammer.metrics.core.MetricsRegistry.newGauge(MetricsRegistry.java:79)  
97:     at com.yammer.metrics.Metrics.newGauge(Metrics.java:70)  
98:     at org.apache.cassandra.metrics.ConnectionMetrics.<init>(ConnectionMetrics.java:78)  
99:     at org.apache.cassandra.net.OutboundTcpConnectionPool.<init>(OutboundTcpConnectionPool.java:53)  
100:     at org.apache.cassandra.net.MessagingService.getConnectionPool(MessagingService.java:493)  
101:     at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:507)  
102:     at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:640)  
103:     at org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:614)  
104:     at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:59)  
105:     at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)  
106:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
107:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
108:     at java.lang.Thread.run(Thread.java:745)  
This is indeed nothing to worry about. When you done upgrading all your nodes in the cluster, do another round of restart, this type of warning logging will disappear.