Why Vagrant
Vagrant provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility of you and your team.


Vagrant in windows
1. Install VM VirtureBox and open the VM Manager
--VM download link: https://www.virtualbox.org/wiki/Downloads
2. Download vagrant for windows
--Vagrant download link: http://downloads.vagrantup.com/
3. Open GitHub, clone the vagrant image to your local repositories
4. Open a shell from the directory /vagrant, and run command: $vagrant init
5. Run command: $vagrant up
--This will automatically generate VM box to your local machine with Vagrant image

PERL MODULE

TAP::Harness
Description: This is a simple test harness which allows tests to be run and results automatically aggregated and output to STDOUT.
Synopsis: 

use TAP::Harness;
my $harness = TAP::Harness->new( \%args );
$harness->runtests(@tests);

Methods:

 my %args = (
    verbosity => 1,
    lib     => [ 'lib', 'blib/lib', 'blib/arch' ],
 )
 my $harness = TAP::Harness->new( \%args );

Verbosity Level:

 1   verbose        Print individual test results to STDOUT.
 0   normal
 -1   quiet          Suppress some test output (mostly failures 
                        while tests are running).
 -2   really quiet   Suppress everything but the tests summary.
 -3   silent         Suppress everything.

DBI
The DBI is the standard database interface module for Perl. It defines a set of methods, variables and conventions that provide a consistent database interface independent of the actual database being used.

WWW::Scripter - For scripting web sites that have scripts
Description: This is a subclass of WWW::Mechanize that uses the W3C DOM and provides support for scripting.
Synopsis:

use WWW::Scripter;
  $w = new WWW::Scripter;

  $w->use_plugin('Ajax');  # packaged separately
  
  $w->get('http://some.site.com/that/relies/on/ajax');
  $w->eval(' alert("Hello from JavaScript") ');
  $w->document->getElementsByTagName('div')->[0]->....

  $w->content; # returns the HTML content, possibly modified
               # by scripts

HTML::TreeBuilder::XPath - add XPath support to HTML::TreeBuilder
Description: This module adds typical XPath methods to HTML::TreeBuilder, to make it easy to query a document.
Synopsis:

use HTML::TreeBuilder::XPath;
  my $tree= HTML::TreeBuilder::XPath->new;
  $tree->parse_file( "mypage.html");
  my $nb=$tree->findvalue( '/html/body//p[@class="section_title"]/span[@class="nb"]');
  my $id=$tree->findvalue( '/html/body//p[@class="section_title"]/@id');

  my $p= $html->findnodes( '//p[@id="toto"]')->[0];
  my $link_texts= $p->findvalue( './a'); # the texts of all a elements in $p
  $tree->delete; # to avoid memory leaks, if you parse many HTML documents 

Date::Calc - Gregorian calendar date calculations

Test::Simple - Basic utilities for writing tests.
Description: This is an extremely simple, extremely basic module for writing tests suitable for CPAN modules and other pursuits. If you wish to do more complicated testing, use the Test::More module (a drop-in replacement for this one).
Synopsis:

 use Test::Simple tests => 1;

  ok( $foo eq $bar, 'foo is bar' );

The basic unit of Perl testing is the ok. For each thing you want to test your program will print out an "ok" or "not ok" to indicate pass or fail. You do this with the ok() function (see below).

The only other constraint is you must pre-declare how many tests you plan to run. This is in case something goes horribly wrong during the test and your test program aborts, or skips a test or whatever. You do this like so:

    use Test::Simple tests => 23;

You must have a plan.

ok

  ok( $foo eq $bar, $name );
  ok( $foo eq $bar );

ok() is given an expression (in this case $foo eq $bar). If it's true, the test passed. If it's false, it didn't. That's about it.
ok() prints out either "ok" or "not ok" along with a test number (it keeps track of that for you).

Test::More - yet another framework for writing test scripts
Description: The purpose of this module is to provide a wide range of testing utilities. Various ways to say "ok" with better diagnostics, facilities to skip tests, test future features and compare complicated data structures. While you can do almost anything with a simple ok() function, it doesn't provide good diagnostic output.

Before anything else, you need a testing plan. This basically declares how many tests your script is going to run to protect against premature failure.

The preferred way to do this is to declare a plan when you use Test::More.

  use Test::More tests => 23;

There are cases when you will not know beforehand how many tests your script is going to run. In this case, you can declare your tests at the end.

  use Test::More;

  ... run your tests ...

  done_testing( $number_of_tests_run );

Sometimes you really don't know how many tests were run, or it's too difficult to calculate. In which case you can leave off $number_of_tests_run.

In some cases, you'll want to completely skip an entire testing script.

  use Test::More skip_all => $skip_reason;

Your script will declare a skip with the reason why you skipped and exit immediately with a zero (success). 

If you want to control what functions Test::More will export, you have to use the 'import' option. For example, to import everything but 'fail', you'd do:

  use Test::More tests => 23, import => ['!fail'];

Alternatively, you can use the plan() function. Useful for when you have to calculate the number of tests.

  use Test::More;
  plan tests => keys %Stuff * 3;

or for deciding between running the tests at all:

  use Test::More;
  if( $^O eq 'MacOS' ) {
      plan skip_all => 'Test irrelevant on MacOS';
  }
  else {
      plan tests => 42;
  }

done_testing

    done_testing();
    done_testing($number_of_tests);

If you don't know how many tests you're going to run, you can issue the plan when you're done running tests.
$number_of_tests is the same as plan(), it's the number of tests you expected to run. You can omit this, in which case the number of tests you ran doesn't matter, just the fact that your tests ran to conclusion.
This is safer than and replaces the "no_plan" plan.

Linux / Unix Command: crond

cron - daemon to execute scheduled commands

Description:

Cron should be started from /etc/rc or /etc/rc.local.

Cron searches /var/spool/cron for crontab files which are named after accounts in /etc/passwd; crontabs found are loaded into memory. Cron also searches for /etc/crontab and the files in the /etc/cron.d/ directory, which are in a different format. Cron then wakes up every minute, examining all stored crontabs, checking each command to see if it should be run in the current minute. When executing commands, any output is mailed to the owner of the crontab (or to the user named in the MAILTO environment variable in the crontab, if such exists).

Additionally, cron checks each minute to see if its spool directory's modtime (or the modtime on /etc/crontab) has changed, and if it has, cron will then examine the modtime on all crontabs and reload those which have changed. Thus cron need not be restarted whenever a crontab file is modified.

Cron (a Linux process that performs background work, often at night) is set up by default on your RedHat system. So you don't have to do anything about it unless you would like to add some tasks to be performed on your system on a regular basis or change the time at which cron performs its duties.

Please note that some of the cron work might be essential for your system functioning properly over a long period of time. Among other things cron may:

- rebuild the database of files which is used when you search for files with the locate command,

- clean the /tmp directory,

- rebuild the manual pages,

- "rotate" the log files, i.e. discard the oldest log files, rename the intermediate logs, and create new logs,

- perform some other checkups, e.g. adding fonts that you recently copied to your system.

Therefore, it may not be the best idea to always switch your Linux machine off for the night--in such a case cron will never have a chance to do its job. If you do like switching off your computer for the night, you may want to adjust cron so it performs its duties at some other time.

To find out when cron wakes up to perform its duties, have a look at the file /etc/crontab, for example:

cat /etc/crontab

It may contain something like this:

# run-parts

01 * * * * root run-parts /etc/cron.hourly

02 4 * * * root run-parts /etc/cron.daily

22 4 * * 0 root run-parts /etc/cron.weekly

42 4 1 * * root run-parts /etc/cron.monthly

You can see that there are four categories of cron jobs: performed hourly, daily, weekly and monthly. You can modify those or add your own category. Here is how it works.

The columns in the entries show: minute (0-59), hour (0-23), day of month (1-31), month of year (1-12), day of week (0-6--Sunday to Saturday). The "*" means "any valid value".

Thus, in the example quoted, the hourly jobs are performed every time the computer clock shows "and one minute", which happens every hour, at one minute past the hour. The daily jobs are performed every time the clock shows 2 minutes past 4 o'clock, which happens once a day. The weekly jobs are performed at 22 minutes past four o'clock in the morning on Sundays. The monthly jobs are performed 42 minutes past four o'clock on the first day of every month. The directory with the script file that contain the command(s) to be executed is shown as the last entry on each line.

If you wanted your jobs to be performed at noon instead of 4 in the morning, just change the 4s to 12s. Cron wakes up every minute and examines if the /etc/crontab has changed so there is no need to re-start anything after you make your changes.

If you wanted to add a job to your cron, place a script which runs your job (or a link to your script) in the directory /etc/cron.hourly or cron.daily or /etc/cron.weekly, or /etc/cron.monthly .

Here is an example of an entry in /etc/crontab which causes a job to be performed three times a week (Mon, Wed, Fri):

02 4 * * 1,3,5 root run-parts/etc/cron.weekly

Ubuntu Linux OpenSSH Server installation and configuration

OpenSSH is a FREE version of the SSH connectivity tools that technical users of the Internet rely on. Users of telnet, rlogin, and ftp may not realize that their password is transmitted across the Internet unencrypted, but it is. OpenSSH encrypts all traffic (including passwords) to effectively eliminate eavesdropping, connection hijacking, and other attacks. Additionally, OpenSSH provides secure tunneling capabilities and several authentication methods, and supports all SSH protocol versions.

Install:

# sudo apt-get install openssh-server openssh-client

Diagnostic Report

ps -ef

List of all running processes

top

List the top resource consumers

env

List all environment variables

find -ls

Recursively list all files and directories

df -hk

Display the file system information

uname -Xa

List information about the server

psrinfo -v

List information about the servers processors

jstack

Get a stack trace of Java program to see what logic is running or stuck in.
Use kill -3 if jstack not working due to high cpu usage

http://download.oracle.com/javase/6/docs/technotes/tools/share/jstack.html

jinfo

Get all Java runtime parameters such as command line options, classpath, etc.

http://download.oracle.com/javase/6/docs/technotes/tools/share/jinfo.html

jmap

Dump Java program memory into file for memory leak analysis.

http://download.oracle.com/javase/6/docs/technotes/tools/share/jmap.html

Data integration

Data integration: retrieve data from different sources and assemble it in a unified way.

Data integration focuses mainly on databases. A database is an organized collection of data. It's similar to a file system, which is an organizational structure for files so they're easy to find, access and manipulate.

There's the common data storage method, also known as data warehousing. Using this method, all the data from the various databases you intend to integrate are extracted, transformed and loaded. That means that the data warehouse first pulls all the data from the various data sources. Then, the data warehouse converts all the data into a common format so that one set of data is compatible with another. Then it loads this new data into its own database. When you submit your query, the data warehouse locates the data, retrieves it and presents it to you in an integrated view.

A data warehouse is a database that stores information from other databases using a common format.

Descriptions of data are called metadata. Metadata is useful for naming and defining data as well as describing the relationship of one set of data to other sets. Data integration systems use metadata to locate the information relevant to queries.

The warehouse must have a database large enough to store data gathered from multiple sources. Some data warehouses include an additional step called a data mart. The data warehouse takes over the duties of aggregating data, while the data mart responds to user queries by retrieving and combining the appropriate data from the warehouse.

One problem with data warehouses is that the information in them isn't always current. That's because of the way data warehouses work -- they pull information from other databases periodically. If the data in those databases changes between extractions, queries to the data warehouse won't result in the most current and accurate views. If the data in a system rarely changes, this isn't a big deal. For other applications, though, it's problematic.

Load balancing

In networking, load balancing is a technique to distribute workload evenly across two or more computers, network links, CPUs, hard drives, or other resources, in order to get optimal resource utilization, maximize throughput, minimize response time, and avoid overload. Using multiple components with load balancing, instead of a single component, may increase reliability through redundancy. The load balancing service is usually provided by a dedicated program or hardware device (such as a multilayer switch or a DNS server).

F5 Networks originally manufactured and sold some of the very first load balancing product, called BIG-IP. If a server went down or became overloaded, BIG-IP directed traffic away from that server to other servers that could handle the load.

F5's BIG-IP product is based on a network appliance (either virtual or physical), which runs F5's Traffic Management Operating System (TMOS), which runs on top of Linux. This appliance can then run one or more product modules (depending on the appliance selected), which provide the BIG-IP functionality.

LDAP

What is LDAP ?

• Lightweight Directory Access Protocol
• Based on X.500
• Directory service (RFC1777)
• Stores attribute based data
• Data generallly read more than written to
  • No transactions
  • No rollback
• Hierarchical data structure
• Entries are in a tree-like structure called Directory Information Tree (DIT)
  

Attribute abbreviations

uid User id

cn Common Name

sn Surname

l Location

ou Organisational Unit

o Organisation

dc Domain Component

st State

c Country

The Lightweight Directory Access Protocol (LDAP) is an application protocol for reading and editing directories over an IP network.

A client starts an LDAP session by connecting to an LDAP server, called a Directory System Agent (DSA), by default on TCP port 389. The client then sends an operation request to the server, and the server sends responses in return. With some exceptions, the client does not need to wait for a response before sending the next request, and the server may send the responses in any order.

A common alternate method of securing LDAP communication is using an SSL tunnel. This is denoted in LDAP URLs by using the URL scheme "ldaps". The default port for LDAP over SSL is 636. The use of LDAP over SSL was common in LDAP Version 2 (LDAPv2) but it was never standardized in any formal specification.

Data integration

Data integration is a set of procedures, techniques, and technologies used to design and build processes that extract, restructure, move, and load data in either operational or analytic data stores either in real time or in batch mode.

Metadata is the “data” about the data; it is the business and technical definitions that provide the

data meaning.

A major function of data integration is to integrate disparate data into a single view of information.

ETL Data integration

ETL is the collection and aggregation of transactional data with data extracted from multiple sources to be conformed into databases used for reporting and analytics.

Most of the cost and maintenance of complex data integration processing occurs in the bulk data

movement space. ETL has experienced explosive growth in both frequency and size in the past 15 years. In the mid-1990s, pushing 30GB to 40GB of data on a monthly basis was considered a

large effort. However, by the twenty-first century, moving a terabyte of data on a daily basis was a requirement. In addition to standard flat file and relational data formats, data integration environments need to consider XML and unstructured data formats. With these new formats, along with the exponential growth of transactional data, multi-terabyte data integration processing environments are not unusual.

ETL: Extract, Transform and Load

ETL stands for extract, transform and load, includes reading data from its source, cleaning it up and formatting it uniformly, and then writing it to the target repository to be exploited.

The data used in ETL processes can come from any source: a mainframe application, an ERP application, a CRM tool, a flat file, an Excel spreadsheet—even a message queue.

The processes enable companies to move data from multiple sources, reformat and cleanse it, and load it into another database, a data mart or a data warehouse for analysis, or on another operational system to support a business process.

In OTC, ETL mostly used in transfer Data from Informix to Vertica, move Data inside database, and extract Data report from database. I will explain in my next blog how OTC use Vertica with market data warehouse.

How to use mysql in Linux

1. sudo apt-get install mysql-client mysql-server
2. sudo apt-get install php5-mysql
3. mysql -u root -p

JDBC Informix

1. Add external jar for jfxjdbc.jar to Java Build Path.
2. Coding:
package db;
import java.io.IOException;
import java.io.InputStream;
import java.sql.*;
import java.util.*;

public class jdbc_informix
{
public static void main(String[] args) throws IOException
{
Connection conn = null;
String userName = "xx";
String password = "xx";
String url = "jdbc:informix-sqli://db.test.ps:1111/table_name:INFORMIXSERVER=table_n";

try
{
Class.forName("com.informix.jdbc.IfxDriver");

System.out.println("Driver OK");

}
catch (Exception e)
{
System.out.println("FAILED: failed to load Informix JDBC driver.");
}

try
{
conn = DriverManager.getConnection(url, userName, password);
System.out.println ("Database connection established");

Statement s = conn.createStatement ();

s.executeQuery ("select * from table_name");

ResultSet rs = s.getResultSet ();
if (rs.next ())
{
String nameVal = rs.getString 1);
System.out.println(nameVal);
}
else
System.out.println("error");

}
catch (SQLException e)
{
System.out.println("FAILED: failed to connect!"+e);
}

}

private static void dispValue(InputStream value) {
// TODO Auto-generated method stub

}
}

File permission in Linux

File Ownership
1. User
2. Group
3. Other

File Permissions
1. Read permission
2. Write permission
3. Execute permission

How to view file permissions
$ls -l
$ ls -l
total 17
drwxr-xr-x 3 nana writers 80 2005-09-20 21:37 dir
-rw-r----- 1 nana writers 8187 2005-09-19 13:35 file
-rwxr-xr-x 1 nana writers 10348 2005-07-17 20:31 otherfile

d = directory
- = regular file
l = symbolic link
s = Unix domain socket
p = named pipe
c = character device file
b = block device file

Set file permissions - symbolic mode

Which user?
u	user/owner
g	group
o	other
a	all
What to do?
+	add this permission
-	remove this permission
=	set exactly this permission
Which permissions?
r	read
w	write
x	execute

First, you decide if you set permissions for the user (u), the group (g), others (o), or all of the three (a). Then, you either add a permission (+), remove it (-), or wipe out the previous permissions and add a new one (=). Next, you decide if you set the read permission (r), write permission (w), or execute permission (x). Last, you'll tell chmod which file's permissions you want to change.
eg. $ chmod g+x testfile
Add execute permissions for group.

Set file permissions - numeric mode
4 = read (r)
2 = write (w)
1 = execute (x)
0 = no permission (-)

$ chmod 755 testfile
equals to: -rwxr-xr-x
$ chmod 640 testfile
equals to: -rw-r-----



Quickly add your public key to an authorized keys file

This will add your public ssh key to an authorized keys file on a remote server for passwordless login.
1. Generate key on local machine:

ssh-keygen -t dsa

In your local machine:
$cd .ssh
$ls
authorized_keys id_dsa id_dsa.pub id_rsa id_rsa.pub known_hosts
$more id_dsa.pub

2. Ensure that the remote server has a .ssh directory
$ cd ~/.ssh
$ ls
authorized_keys id_rsa id_rsa.pub known_hosts
$vi authorized_keys

3. Add your public key into remote server.
Now you can ssh to the remote server without entering your password.
Keep in mind that all someone needs to login to the remote server, is the file on your local machine ~/.ssh/id_rsa, so make sure it is secure.

More bash shell commands

1. The basic output shows the process ID (PID) of the programs, the terminal (TTY) that they are
running from, and the CPU time the process has used.
$ ps
PID TTY TIME CMD
3081 pts/0 00:00:00 bash
3209 pts/0 00:00:00 ps

2. see everything running on the system
$ ps -ef

3. The kill command allows you to send signals to processes based on their process ID (PID).
$ kill 3940

4. see how much disk space is available on an individual device. The df command allows us to easily see what’s happening on all of the mounted disks
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 6813412 2889540 3577760 45% /
none 1024880 212 1024668 1% /dev
none 1030476 644 1029832 1% /dev/shm
none 1030476 96 1030380 1% /var/run
none 1030476 0 1030476 0% /var/lock

$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 6.5G 2.8G 3.5G 45% /
none 1001M 212K 1001M 1% /dev
none 1007M 644K 1006M 1% /dev/shm
none 1007M 96K 1007M 1% /var/run
none 1007M 0 1007M 0% /var/lock

5. The du command shows the disk usage for a specific directory (by default, the current directory). This is a quick way to determine if you have any obvious disk hogs on the system.
t$ du
8 ./testfold
116 .

$ du -c
8 ./testfold
116 .
116 total

$ du -h
8.0K ./testfold
116K .

$ du -s
116 .

6. sort
$ cat test8
8
93
5
6
28
$ sort test8
28
5
6
8
93
$ sort -n test8
5
6
8
28
93

$ sort -M file3
The -n parameter is great for sorting numerical outputs, such as the output of the du command:
$ du -sh * | sort -nr

7.
$ cat > file1
one
two
three
four
five
$ grep three file1
three
$ grep t file1
two
three
$ grep -v t file1
one
four
five
$ grep -n t file1
2:two
3:three
$ grep -c t file1
2
$ grep -e t -e f file1
two
three
four
five
$ grep [tf] file1
two
three
four
five

8. Compressing data
■ bzip2 for compressing files
■ bzcat for displaying the contents of compressed text files
■ bunzip2 for uncompressing compressed .bz2 files
■ bzip2recover for attempting to recover damaged compressed files

$ ls -l
total 4
-rw-r--r-- 1 linna linna 9 2011-02-24 02:20 test1
$ bzip2 test1
$ ls -l
total 4
-rw-r--r-- 1 linna linna 45 2011-02-24 02:20 test1.bz2
$ more test1.bz2
BZh91AY&SYøG²
$ bzcat test1.bz2
test
test
$ bunzip2 test1.bz2
$ ls -l
total 4
-rw-r--r-- 1 linna linna 9 2011-02-24 02:20 test1
$ more test1
test
test

9. The gzip utility
■ gzip for compressing files
■ gzcat for displaying the contents of compressed text files
■ gunzip for uncompressing files

10. Archiving data
By far the most popular archiving tool used in Unix and Linux is the tar command.

File Handling in shell

1. Create file with unique inode number
$ touch test1
$ ls -il test1
1954793 -rw-r--r-- 1 rich rich 0 Sep 1 09:35 test1

2. Create file without inode number
$ touch test1
$ ls -l test1
-rw-r--r-- 1 rich rich 0 Sep 1 09:37 test1

3. Change the file modification time
$ touch -t 200812251200 test1
$ ls -l test1
-rw-r--r-- 1 rich rich 0 Dec 25 2008 test1

4. Copy files
$cp source destination

5. Copy file to a directory
$ cp test1 dir1
To copy a file to the current directory you’re in, you can use the dot symbol.

6. Copy the contents of an entire directory in one command:
$ cp -R dir1 dir2

7. Linking files
If you need to maintain two (or more) copies of the same file on the system, instead of having separate physical copies, you can use one physical copy and multiple virtual copies, called links. A link is a placeholder in a directory that points to the real location of the file.
Hard link: create a separate file that contains information about the original file(test1) and where to locate it. After you create it, you will see the inode for test4 is the same as test1.
$ cp -l test1 test4
$ ls -il
Soft link: the -s parameter creates a symbolic, or soft link.
Tip: Instead of using the cp command, if you want to link files you can also use the ln command. By default the ln command creates hard links. If you want to create a soft link, you’ll still need to use the -s parameter.

Remember that the hard link file uses the same inode number as the original file. The hard link
file maintains that inode number until you remove the last linked file, preserving the data! All the soft link file knows is that the underlying file is now gone, so it has nothing to point to.

8. Renaming files
$ mv test2 test6
$ ls -il test*
Rename test2 to test6. Notice that moving the file changed the filename but kept the same inode number and the timestamp value.

9. Renaming directory
$ mv dir2 dir4

10. Deleting files
$ rm -i test2
rm: remove `test2’? y
$ ls -l

11. Creating directories
$ mkdir dir3
$ ls -il

12. Deleting directories
$ rmdir dir3
$ rmdir dir1
rmdir: dir1: Directory not empty
if you really want to remove a directory, you can use the -r parameter to recursively
remove the directory.

13. Viewing file statistics: The results from the stat command show just about everything you’d want to know about the file being examined:
$ stat test10

14. Viewing the file type: The file command is a handy little utility to have around. It has the ability to peek inside of a file and determine just what kind of file it is:
$ file test1
test1: ASCII text

15. Viewing the whole file
$ cat test1
$ cat -n test1
$ cat -b test1
$ cat -s test1
$ cat -T test1
$ more test1
The more command displays a text file, but stops after it displays each page of data.
The more command allows some rudimentary movement through the text file. For more advanced features, try the less command.
$ less test1

16. Viewing parts of a file
$tail -f
A great way to monitor the system log file in real-time mode.

How to write shell script

1. Type following cat command:
$ cat > first
#
# My first shell script
#
clear
echo "Knowledge is Power"
echo -e "An apple a day keeps away \a\t\tdoctor\n"
$ echo "Today is date"
$ echo "Today is `date`"
Press Ctrl + D to save. Now our script is ready.
2. Set Execute permission for our script first:
$ chmod +x first
3. Run the shell script
$ ./first
Or: $ bash first
Or: $ /bin/sh first

Addition:
1. Type bc at $ prompt to start Linux calculator program, quit to exit
$ bc
After this command bc is started and waiting for you commands, i.e. give it some calculation as
follows type 5 + 2 as
5 + 2
7
Now what happened if you type 5 > 2 as follows
5 < 2
0
0 (Zero) is response of bc, false
2. eg.
$ cat > demo
#!/bin/sh
#
# Script that demos, command line args
#
echo "Total number of command line argument are $#"
echo "$0 is script name"
echo "$1 is first argument"
echo "$2 is second argument"
echo "All of them are :- $*"
executable & run:
$ chmod +x demo
$ ./demo Hello World
3. eg. show file
$ cat > showfile
#!/bin/sh
#
#Script to print file
#
if cat $1
then
echo -e "\n\nFile $1, found and successfully echoed"
fi
Now run it.
$ chmod +x showfile
$./showfile foo
4. eg. delete file
#
# Script to test rm command and exist status
#
if rm $1
then
echo "$1 file deleted"
fi
(Press Ctrl + d to save)
$ chmod +x trmif
$ ./trmfi foo
5. eg. for loop
$ cat > testfor
for i in 1 2 3 4 5
do
echo "Welcome $i times"
done
Run it as,
$ chmod +x testfor
$ ./testfor
6. eg.
n=$1
for i in 1 2 3 4 5 6 7 8 9 10
do
echo "$n * $i = `expr $i \* $n`"
done
Save and Run it as
$ chmod +x mtable
$ ./mtable 7
7. eg. the read statement
$ cat > sayH
#
#Script to read your name from key-board
#
echo "Your first name please:"
read fname
echo "Hello $fname, Lets be friend!"
Run it as follows
$ chmod +x sayH
$ ./sayH
8. eg.
To run two command with one command line.For eg. $ date;who ,Will print today's date followed by users who are currently login.
9. eg. function
$ SayHello()
{
echo "Hello $LOGNAME, Have nice computing"
return
}
Now to execute this SayHello() function just type it name as follows
$ SayHello

Shell

The shell allows you to chain commands together into a single step. You can use semicolon and put both commands on the same line, but in a shell script, you can list commands on separate lines. The shell will process commands in the order in which they appear in the file.

Creating a Script File

1. use a text editor to create a file, then enter the command to the file

2. specify the shell you are using, format: #!/bin/bash

3. give permission to execute the file: $chmod u+x test

4. $./test

Structured Command:
1. if -then
$ cat test1
#!/bin/bash
# testing the if statement
if date
then
echo "it worked"
fi

$ ./test1
Thu Feb 24 15:29:12 EST 2011
it worked

Shell is a user program or it's environment provided for user interaction. Shell is an command language interpreter that executes commands read from the standard input device (keyboard) or from a file.

Shell is not part of system kernel, but uses the system kernel to execute programs, create files etc.

Several shell available with Linux including: BASH, CSH, KSH, TCSH

Tip: To find all available shells in your system type following command:
$ cat /etc/shells

Tip: To find your current shell type following command
$ echo $SHELL

Normally shells are interactive. It means shell accept command from you (via keyboard) and execute them. But if you use command one by one (sequence of 'n' number of commands) , then you can store this sequence of command to text file and tell the shell to execute this text file instead of entering the commands. This is know as shell script.

Following steps are required to write shell script:

(1) Use any editor like vi or mcedit to write shell script.

(2) After writing shell script set execute permission for your script as follows
syntax:
chmod permission your-script-name

Examples:
$ chmod +x your-script-name
$ chmod 755 your-script-name

Note: This will set read write execute(7) permission for owner, for group and other permission is read and execute only(5).

(3) Execute your script as
syntax:
bash your-script-name
sh your-script-name
./your-script-name

Examples:
$ bash bar
$ sh bar
$ ./bar

NOTE In the last syntax ./ means current directory, But only . (dot) means execute given command file in current shell without starting the new copy of shell, The syntax for . (dot) command is as follows
Syntax:
. command-name

Debug a script:

$bash -x script_name

$bash -xv script_name

cd to /etc/init.d and view various system init scripts:

cd /etc/init.d
ls
vi ssh

Display your current PATH:

echo $PATH

Sample outputs:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Customize your PATH variable and remove /usr/games from PATH:

export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export PATH=$PATH:/usr/games

The whatis command is used display a short description about command. whatis command searches the manual page names and displays the manual page descriptions for a command:

whatis command-name
whatis date
whatis ifconfig
whatis ping

Change password:
$passwd

Create alias:
alias name='command'
alias name='command arg1 arg2'

Remove alias:
unalias alias-name

Go to root: $su

alias update='apt-get update && apt-get upgrade'

Doing logout and login back operation: $bash

Basic Linux Command

Command	Example	Description
cat		Sends file contents to standard output. This is a way to list the contents of short files to the screen. It works well with piping.
	cat .bashrc	Sends the contents of the ".bashrc" file to the screen.
cd		Change directory
	cd /home	Change the current working directory to /home. The '/' indicates relative to root, and no matter what directory you are in when you execute this command, the directory will be changed to "/home".
	cd httpd	Change the current working directory to httpd, relative to the current location which is "/home". The full path of the new working directory is "/home/httpd".
	cd ..	Move to the parent directory of the current directory. This command will make the current working directory "/home.
	cd ~	Move to the user's home directory which is "/home/username". The '~' indicates the users home directory.
cp		Copy files
	cp myfile yourfile	Copy the files "myfile" to the file "yourfile" in the current working directory. This command will create the file "yourfile" if it doesn't exist. It will normally overwrite it without warning if it exists.
	cp -i myfile yourfile	With the "-i" option, if the file "yourfile" exists, you will be prompted before it is overwritten.
	cp -i /data/myfile .	Copy the file "/data/myfile" to the current working directory and name it "myfile". Prompt before overwriting the file.
	cp -dpr srcdir destdir	Copy all files from the directory "srcdir" to the directory "destdir" preserving links (-p option), file attributes (-p option), and copy recursively (-r option). With these options, a directory and all it contents can be copied to another directory.
dd	dd if=/dev/hdb1 of=/backup/	Disk duplicate. The man page says this command is to "Convert and copy a file", but although used by more advanced users, it can be a very handy command. The "if" means input file, "of" means output file.
df		Show the amount of disk space used on each mounted filesystem.
less	less textfile	Similar to the more command, but the user can page up and down through the file. The example displays the contents of textfile.
ln		Creates a symbolic link to a file.
	ln -s test symlink	Creates a symbolic link named symlink that points to the file test Typing "ls -i test symlink" will show the two files are different with different inodes. Typing "ls -l test symlink" will show that symlink points to the file test.
locate		A fast database driven file locator.
	slocate -u	This command builds the slocate database. It will take several minutes to complete this command. This command must be used before searching for files, however cron runs this command periodically on most systems.
	locate whereis	Lists all files whose names contain the string "whereis".
logout		Logs the current user off the system.
ls		List files
	ls	List files in the current working directory except those starting with . and only show the file name.
	ls -al	List all files in the current working directory in long listing format showing permissions, ownership, size, and time and date stamp
more		Allows file contents or piped output to be sent to the screen one page at a time.
	more /etc/profile	Lists the contents of the "/etc/profile" file to the screen one page at a time.
	ls -al |more	Performs a directory listing of all files and pipes the output of the listing through more. If the directory listing is longer than a page, it will be listed one page at a time.
mv		Move or rename files
	mv -i myfile yourfile	Move the file from "myfile" to "yourfile". This effectively changes the name of "myfile" to "yourfile".
	mv -i /data/myfile .	Move the file from "myfile" from the directory "/data" to the current working directory.
pwd		Show the name of the current working directory
	more /etc/profile	Lists the contents of the "/etc/profile" file to the screen one page at a time.
shutdown		Shuts the system down.
	shutdown -h now	Shuts the system down to halt immediately.
	shutdown -r now	Shuts the system down immediately and the system reboots.
whereis		Show where the binary, source and manual page files are for a command
	whereis ls	Locates binaries and manual pages for the ls command.

Linux Command Related with Process

Following tables most commonly used command(s) with process:

For this purpose	Use this Command	Examples*
To see currently running process	ps	$ ps
To stop any process by PID i.e. to kill process	kill {PID}	$ kill 1012
To stop processes by name i.e. to kill process	killall {Process-name}	$ killall httpd
To get information about all running process	ps -ag	$ ps -ag
To stop all process except your shell	kill 0	$ kill 0
For background processing (With &, use to put particular command and program in background)	linux-command &	$ ls / -R | wc -l &
To display the owner of the processes along with the processes	ps aux	$ ps aux
To see if a particular process is running or not. For this purpose you have to use ps command in combination with the grep command	ps ax | grep process-U-want-to see	For e.g. you want to see whether Apache web server process is running or not then give command $ ps ax | grep httpd
To see currently running processes and other information like memory and CPU usage with real time updates.	top See the output of top command.	$ top Note that to exit from top command press q.
To display a tree of processes	pstree	$ pstree

* To run some of this command you need to be root or equivalnt user.