Tuesday, September 29, 2009

sh^M: bad interpreter: No such file or directory


$ ./configure
bash: ./configure: /bin/sh^M: bad interpreter: No such file or directory

Problem is that we r trying to run script saved in dos format at some point.
Convert this to unix by using dos2unix

dos2unix is contained in package tofrodos

sudo apt-get install tofrodos
dos2unix configure

It will (attempt) to convert this/these files properly to a use-able format

Monday, September 7, 2009

How To Enable Adobe's Flash Player In Google Chrome (Ubuntu 9.04)

If the Flash plugin is already installed for Firefox, we have to find out where it is located:

sudo updatedb
sudo locate libflashplayer.so

If it is installed, output should be similar to this one...

falko@falko-desktop:~$ sudo locate libflashplayer.so
/opt/Adobe AIR/Versions/1.0/Resources/libflashplayer.so
/usr/lib/flashplugin-installer/libflashplayer.so
falko@falko-desktop:~$

... which means the Flash plugin is located in /usr/lib/flashplugin-installer/libflashplayer.so. If there's no output, this means that the Flash plugin isn't installed - you can then install it as follows:

sudo aptitude install flashplugin-installer

Now we create a plugins directory for Google Chrome...

sudo mkdir /opt/google/chrome/plugins

... and copy the Flash plugin to it:

sudo cp /usr/lib/flashplugin-installer/libflashplayer.so /opt/google/chrome/plugins

Finally we have to tell the Google Chrome launcher that it should look out for plugins. To do this, we right-click on Applications and select Edit Menus:




In the Menu Editor, go to Internet > Google Chrome and click on the Properties button:




In the Launcher Properties window, replace the contents of the Command field with this line...

/opt/google/chrome/google-chrome --enable-plugins %U

... and click on Close:

Then leave the Menu Editor.

That's it! We can now launch Google Chrome (Applications > Internet > Google Chrome):





Tuesday, August 25, 2009

Seting-Up a HTTP Proxy Server with Authentication and Filtering

sudo apt-get install squid
Configure Squid by opening /etc/squid/squid.conf using your favorite text editor. In the configuration file, search for the following directives and modify (or add, if they don't exist) as it follows:

http_port 3128 - The port Squid will listen for connections. If your system has two or more interfaces, you can specify which IP address to use. Eg: http_port 192.168.0.1:3128

http_access deny all - Search for it in the config file, uncomment it (remove the # in front), and replace deny with allow so it becomes http_access allow all.

Restart the Squid proxy with:
CODE
$ sudo /etc/init.d/squid restart

Now you should have a fully functional HTTP proxy. To try it out, open a browser, open its preferences dialog and go to proxy settings. Here, enter the IP address of the machine running Squid and the port set in squid.conf. Now load a webpage.

SETTING UP SQUID AUTHENTICATION AND WEB FILTERING

This section will allow you to set up a web site filter for kids. The first time an address is entered in the browser's address bar, an authentication dialog will pop-up, prompting for a username and password. We will set-up two usernames, one with full and another with restricted access.

First, open the /etc/squid/squid.conf and add the following line in the auth_param section:

auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/passwd

Now create the user accounts using htpasswd (use -c only for the first user):
CODE
$ sudo htpasswd -c /etc/squid/passwd dad
Enter a password for user 'dad':
Again:

$ sudo htpasswd /etc/squid/passwd kid
Another password:
Again:

Create the ACLs by adding the following lines in the ACCESS CONTROLS (acl) sections in Squid.conf:
CODE
acl dadUser proxy_auth dad
acl kidUser proxy_auth kid
acl whitelist dstdomain "/etc/squid/whitelist"
http_access allow dadUser
http_access allow kidUser whitelist

Create the whitelist by opening a text editor, adding allowed domains like this:
.google.com
.kids-play.com
.yahoo.com
.msn.com


and save it as /etc/squid/whitelist.

Finally, search for http_access allow all in the Squid config file and modify it so it looks like this:
http_access deny all

This is how my Squid config sections look like:
CODE
# NETWORK OPTIONS
# Squid normally listens to port 3128
http_port 192.168.0.1:3128

# TAG: auth_param
#Recommended minimum configuration per scheme:
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/passwd

# ACCESS CONTROLS
# TAG: acl
acl dadUser proxy_auth dad
acl kidUser proxy_auth kid
acl whitelist dstdomain "/etc/squid/whitelist"
http_access allow dadUser
http_access allow kidUser whitelist

# TAG: http_access
# And finally deny all other access to this proxy
http_access deny all

Use deny all for squid with authentication and allow all for basic squid configuration.

create a shared and a static library with gcc


The code for the library

This is the code that goes into the library. It exhibits one single function that takes two doubles and calculates their mean value and returns it.
calc_mean.c
//#include   double mean(double a, double b) {   return (a+b) / 2; } 

The header file

Of course, we need a header file.
calc_mean.h
double mean(double, double); 

Creating the static library

A static library is basically a set of object files that were copied into a single file. This single file is the static library. The static file is created with the archiver (ar).
First, calc_mean.c is turned into an object file:
gcc -c calc_mean.c -o calc_mean.o 
Then, the archiver (ar) is invoked to produce a static library (named libmean.a) out of the object file calc_mean.o.
ar  rcs libmean.a      calc_mean.o 
Note: the library must start with the three letters lib and have the suffix .a.

Creating the shared library

As with static libraries, an object file is created. The -fPIC option tells gcc to create position independant code which is necessary for shared libraries. Note also, that the object file created for the static library will be overwritten. That's not bad, however, because we have a static library that already contains the needed object file.
gcc -c -fPIC calc_mean.c -o calc_mean.o     
For some reason, gcc says:
cc1: warning: -fPIC ignored for target (all code is position independent) 
It looks like -fPIC is not necessary on x86, but all manuals say, it's needed, so I use it too.
Now, the shared library is created
gcc -shared -Wl,-soname,libmean.so.1 -o libmean.so.1.0.1  calc_mean.o 
Note: the library must start with the three letter lib.

The programm using the library

This is the program that uses the calc_mean library. Once, we will link it against the static library and once against the shared library.
main.c
#include  #include "calc_mean.h"  int main(int argc, char* argv[]) {    double v1, v2, m;   v1 = 5.2;   v2 = 7.9;    m  = mean(v1, v2);    printf("The mean of %3.2f and %3.2f is %3.2f\n", v1, v2, m);    return 0; } 

Linking against static library

gcc -static main.c -L. -lmean -o statically_linked 
Note: the first three letters (the lib) must not be specified, as well as the suffix (.a)

Linking against shared library

gcc main.c -o dynamically_linked -L. -lmean 
Note: the first three letters (the lib) must not be specified, as well as the suffix (.so)

Executing the dynamically linked programm

LD_LIBRARY_PATH=. ./dynamically_linked 

Creating a library with the GNU archiver

Creating a library with the GNU archiver

The GNU archiver ar combines a collection of object files into a single archive file, also known as a library. An archive file is simply a convenient way of distributing a large number of related object files together

To demonstrate the use of the GNU archiver we will create a small library ‘libhello.a’ containing two functions hello and bye.

The first object file will be generated from the source code for the hello function, in the file ‘hello_fn.c’ seen earlier:

#include  #include "hello.h"  void  hello (const char * name) {   printf ("Hello, %s!\n", name); } 

The second object file will be generated from the source file ‘bye_fn.c’, which contains the new function bye:

#include  #include "hello.h"  void  bye (void) {   printf ("Goodbye!\n"); } 

Both functions use the header file ‘hello.h’, now with a prototype for the function bye():

void hello (const char * name); void bye (void); 

The source code can be compiled to the object files ‘hello_fn.o’ and ‘bye_fn.o’ using the commands:

$ gcc -Wall -c hello_fn.c $ gcc -Wall -c bye_fn.c 

These object files can be combined into a static library using the following command line:

$ ar cr libhello.a hello_fn.o bye_fn.o 

The option cr stands for "create and replace".(33) If the library does not exist, it is first created. If the library already exists, any original files in it with the same names are replaced by the new files specified on the command line. The first argument ‘libhello.a’ is the name of the library. The remaining arguments are the names of the object files to be copied into the library.

The archiver ar also provides a "table of contents" option t to list the object files in an existing library:

$ ar t libhello.a hello_fn.o bye_fn.o 

Note that when a library is distributed, the header files for the public functions and variables it provides should also be made available, so that the end-user can include them and obtain the correct prototypes.

We can now write a program using the functions in the newly created library:

#include "hello.h"  int main (void) {   hello ("everyone");   bye ();   return 0; } 

This file can be compiled with the following command line, as described in section 2.7 Linking with external libraries, assuming the library ‘libhello.a’ is stored in the current directory:

$ gcc -Wall main.c libhello.a -o hello 

The main program is linked against the object files found in the library file ‘libhello.a’ to produce the final executable.

The short-cut library linking option -l can also be used to link the program, without needing to specify the full filename of the library explicitly:

$ gcc -Wall -L. main.c -lhello -o hello 

The option -L. is needed to add the current directory to the library search path. The resulting executable can be run as usual:

$ ./hello Hello, everyone! Goodbye! 

It displays the output from both the hello and bye functions defined in the library.

Debugging GCC

Examining core files

In addition to allowing programs to be run under the debugger, an important benefit of the -g option is the ability to examine the cause of a program crash from a "core dump".

When a program exits abnormally (i.e. crashes) the operating system can write out a core file (usually named ‘core’) which contains the in-memory state of the program at the time it crashed. This file is often referred to as a core dump.(14) Combined with information from the symbol table produced by -g, the core dump can be used to find the line where the program stopped, and the values of its variables at that point.

This is useful both during the development of software and after deployment--it allows problems to be investigated when a program has crashed "in the field".

Here is a simple program containing an invalid memory access bug, which we will use to produce a core file:

int foo (int *p);  int main (void) {   int *p = 0;   /* null pointer */   return foo (p); }  int foo (int *p) {   int y = *p;   return y; } 

The program attempts to dereference a null pointer p, which is an invalid operation. On most systems, this will cause a crash. (15)

In order to be able to find the cause of the crash later, we will need to compile the program with the -g option:

$ gcc -Wall -g null.c 

Note that a null pointer will only cause a problem at run-time, so the option -Wall does not produce any warnings.

Running the executable file on an x86 GNU/Linux system will cause the operating system to terminate the program abnormally:

$ ./a.out  Segmentation fault (core dumped) 

Whenever the error message ‘core dumped’ is displayed, the operating system should produce a file called ‘core’ in the current directory.(16) This core file contains a complete copy of the pages of memory used by the program at the time it was terminated. Incidentally, the term segmentation fault refers to the fact that the program tried to access a restricted memory "segment" outside the area of memory which had been allocated to it.

Some systems are configured not to write core files by default, since the files can be large and rapidly fill up the available disk space on a system. In the GNU Bashshell the command ulimit -c controls the maximum size of core files. If the size limit is zero, no core files are produced. The current size limit can be shown by typing the following command:

$ ulimit -c 0 

If the result is zero, as shown above, then it can be increased with the following command to allow core files of any size to be written:(17)

$ ulimit -c unlimited 

Note that this setting only applies to the current shell. To set the limit for future sessions the command should be placed in an appropriate login file, such as‘.bash_profile’ for the GNU Bash shell.

Core files can be loaded into the GNU Debugger gdb with the following command:

$ gdb EXECUTABLE-FILE CORE-FILE 

Note that both the original executable file and the core file are required for debugging--it is not possible to debug a core file without the corresponding executable. In this example, we can load the executable and core file with the command:

$ gdb a.out core 

The debugger immediately begins printing diagnostic information, and shows a listing of the line where the program crashed (line 13):

$ gdb a.out core Core was generated by `./a.out'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0  0x080483ed in foo (p=0x0) at null.c:13 13        int y = *p; (gdb) 

The final line (gdb) is the GNU Debugger prompt--it indicates that further commands can be entered at this point.

To investigate the cause of the crash, we display the value of the pointer p using the debugger print command:

(gdb) print p $1 = (int *) 0x0 

This shows that p is a null pointer (0x0) of type ‘int *’, so we know that dereferencing it with the expression *p in this line has caused the crash.


Displaying a backtrace

The debugger can also show the function calls and arguments up to the current point of execution--this is called a stack backtrace and is displayed with the commandbacktrace:

(gdb) backtrace #0  0x080483ed in foo (p=0x0) at null.c:13 #1  0x080483d9 in main () at null.c:7 

In this case, the backtrace shows that the crash occurred at line 13 after the function foo was called from main with an argument of p=0x0 at line 7 in ‘null.c’. It is possible to move to different levels in the stack trace, and examine their variables, using the debugger commands up and down.

Setting a breakpoint

A breakpoint stops the execution of a program and returns control to the debugger, where its variables and memory can be examined before continuing. Breakpoints can be set for specific functions, lines or memory locations with the break command.

To set a breakpoint on a specific function, use the command break function-name. For example, the following command sets a breakpoint at the start of the mainfunction in the program above:

$ gdb a.out (gdb) break main Breakpoint 1 at 0x80483c6: file null.c, line 6. 

The debugger will now take control of the program when the function main is called. Since the main function is the first function to be executed in a C program the program will stop immediately when it is run:

(gdb) run Starting program: a.out  Breakpoint 1, main () at null.c:6 6         int *p = 0;   /* null pointer */ (gdb) 

The display shows the line that will be executed next (the line number is shown on the left). The breakpoint stops the program before the line is executed, so at this stage the pointer p is undefined and has not yet been set to zero.


Stepping through the program

To move forward and execute the line displayed above, use the command step:

(gdb) step 7         return foo (p); 

After executing line 6, the debugger displays the next line to be executed. The pointer p will now have been set to zero (null):

(gdb) print p $1 = (int *) 0x0 

The command step will follow the execution of the program interactively through any functions that are called in the current line. If you want to move forward without tracing these calls, use the command next instead.


Modifying variables

To temporarily fix the null pointer bug discovered above, we can change the value of p in the running program using the set variable command.

Variables can be set to a specific value, or to the result of an expression, which may include function calls. This powerful feature allows functions in a program to be tested interactively through the debugger.

In this case we will interactively allocate some memory for the pointer p using the function malloc, storing the value 255 in the resulting location:

(gdb) set variable p = malloc(sizeof(int)) (gdb) print p $2 = (int *) 0x40013f98    (address allocated by malloc) (gdb) set variable *p = 255 (gdb) print *p $3 = 255 

If we now continue stepping through the program with the new value of p the previous segmentation fault will not occur:

(gdb) step foo (p=0x40013f98) at null.c:13 13        int y = *p; (gdb) step 14        return y;


Continuing execution

The command finish continues execution up to the end of the current function, displaying the return value:

(gdb) finish Run till exit from #0  0x08048400 in foo (p=0x40013f98)  at null.c:15 0x080483d9 in main () at null.c:7 7         return foo (p); Value returned is $13 = 255 

To continue execution until the program exits (or hits the next breakpoint) use the command continue,

(gdb) continue Continuing. Program exited with code 0377. 

Note that the exit code is shown in octal (0377 base 8 = 255 in base 10).

Shared libraries and static libraries

External libraries are usually provided in two forms: static libraries and shared libraries. Static libraries are the ‘.a’ files seen earlier. When a program is linked against a static library, the machine code from the object files for any external functions used by the program is copied from the library into the final executable.

Shared libraries are handled with a more advanced form of linking, which makes the executable file smaller. They use the extension ‘.so’, which stands for shared object.

An executable file linked against a shared library contains only a small table of the functions it requires, instead of the complete machine code from the object files for the external functions. Before the executable file starts running, the machine code for the external functions is copied into memory from the shared library file on disk by the operating system--a process referred to as dynamic linking.

Dynamic linking makes executable files smaller and saves disk space, because one copy of a library can be shared between multiple programs. Most operating systems also provide a virtual memory mechanism which allows one copy of a shared library in physical memory to be used by all running programs, saving memory as well as disk space.

Furthermore, shared libraries make it possible to update a library without recompiling the programs which use it (provided the interface to the library does not change).

Because of these advantages gcc compiles programs to use shared libraries by default on most systems, if they are available. Whenever a static library ‘libNAME.a’would be used for linking with the option -lNAME the compiler first checks for an alternative shared library with the same name and a ‘.so’ extension.


It is possible for the system administrator to set the LD_LIBRARY_PATH variable for all users, by adding it to a default login script, such as ‘/etc/profile’. On GNU systems, a system-wide path can also be defined in the loader configuration file ‘/etc/ld.so.conf’.

Alternatively, static linking can be forced with the -static option to gcc to avoid the use of shared libraries:

$ gcc -Wall -static -I/opt/gdbm-1.8.3/include/      -L/opt/gdbm-1.8.3/lib/ dbmain.c -lgdbm 

This creates an executable linked with the static library ‘libgdbm.a’ which can be run without setting the environment variable LD_LIBRARY_PATH or putting shared libraries in the default directories:

Environment variables

The search paths for header files and libraries can also be controlled through environment variables in the shell. These may be set automatically for each session using the appropriate login file, such as ‘.bash_profile’ in the case of GNU Bash.

Additional directories can be added to the include path using the environment variable C_INCLUDE_PATH (for C header files) or CPLUS_INCLUDE_PATH (for C++ header files). For example, the following commands will add ‘/opt/gdbm-1.8.3/include’ to the include path when compiling C programs:

$ C_INCLUDE_PATH=/opt/gdbm-1.8.3/include  $ export C_INCLUDE_PATH 

and similarly for C++ programs:

$ CPLUS_INCLUDE_PATH=/opt/gdbm-1.8.3/include  $ export CPLUS_INCLUDE_PATH 

This directory will be searched after any directories specified on the command line with the option -I, and before the standard default directories (such as‘/usr/local/include’ and ‘/usr/include’). The shell command export is needed to make the environment variable available to programs outside the shell itself, such as the compiler--it is only needed once for each variable in each shell session, and can also be set in the appropriate login file.(8)

Similarly, additional directories can be added to the link path using the environment variable LIBRARY_PATH. For example, the following commands will add ‘/opt/gdbm-1.8.3/lib’ to the link path:

$ LIBRARY_PATH=/opt/gdbm-1.8.3/lib $ export LIBRARY_PATH 

This directory will be searched after any directories specified on the command line with the option -L, and before the standard default directories (such as‘/usr/local/lib’ and ‘/usr/lib’).

With the environment variable settings given above the program ‘dbmain.c’ can be compiled without the -I and -L options,

$ gcc -Wall dbmain.c -lgdbm 

because the default paths now use the directories specified in the environment variables C_INCLUDE_PATH and LIBRARY_PATH. The same compilation command with g++would use the environment variables CPLUS_INCLUDE_PATH and LIBRARY_PATH.


Following the standard Unix convention for search paths, several directories can be specified together in an environment variable as a colon separated list:

DIR1:DIR2:DIR3:... 

The directories are then searched in order from left to right. A single dot ‘.’ can be used to specify the current directory.(9)

For example, the following settings create default include and link paths for packages installed in the current directory ‘.’ and the ‘include’ and ‘lib’ directories under‘/opt/gdbm-1.8.3’ and ‘/net’ respectively:

$ C_INCLUDE_PATH=.:/opt/gdbm-1.8.3/include:/net/include $ LIBRARY_PATH=.:/opt/gdbm-1.8.3/lib:/net/lib

Source :

An Introduction to GCC - for the GNU compilers gcc and g++

Setting search paths

A common problem when compiling a program using library header files is the error:

FILE.h: No such file or directory 

This occurs if a header file is not present in the standard include file directories used by gcc. A similar problem can occur for libraries:

/usr/bin/ld: cannot find library 

This happens if a library used for linking is not present in the standard library directories used by gcc.

By default, gcc searches the following directories for header files:

/usr/local/include/ /usr/include/ 

and the following directories for libraries:

/usr/local/lib/ /usr/lib/ 

The list of directories for header files is often referred to as the include path, and the list of directories for libraries as the library search path or link path.

The directories on these paths are searched in order, from first to last in the two lists above.(7) For example, a header file found in ‘/usr/local/include’ takes precedence over a file with the same name in ‘/usr/include’. Similarly, a library found in ‘/usr/local/lib’ takes precedence over a library with the same name in‘/usr/lib’.

When additional libraries are installed in other directories it is necessary to extend the search paths, in order for the libraries to be found. The compiler options -I and-L add new directories to the beginning of the include path and library search path respectively.

The following example program uses a library that might be installed as an additional package on a system--the GNU Database Management Library (GDBM). The GDBM Library stores key-value pairs in a DBM file, a type of data file which allows values to be stored and indexed by a key (an arbitrary sequence of characters). Here is the example program ‘dbmain.c’, which creates a DBM file containing a key ‘testkey’ with the value ‘testvalue’:

#include  #include   int main (void) {   GDBM_FILE dbf;   datum key = { "testkey", 7 };     /* key, length */   datum value = { "testvalue", 9 }; /* value, length */    printf ("Storing key-value pair... ");   dbf = gdbm_open ("test", 0, GDBM_NEWDB, 0644, 0);   gdbm_store (dbf, key, value, GDBM_INSERT);   gdbm_close (dbf);   printf ("done.\n");   return 0; } 

The program uses the header file ‘gdbm.h’ and the library ‘libgdbm.a’. If the library has been installed in the default location of ‘/usr/local/lib’, with the header file in‘/usr/local/include’, then the program can be compiled with the following simple command:

$ gcc -Wall dbmain.c -lgdbm 

Both these directories are part of the default gcc include and link paths.

However, if GDBM has been installed in a different location, trying to compile the program will give the following error:

$ gcc -Wall dbmain.c -lgdbm     dbmain.c:1: gdbm.h: No such file or directory 

For example, if version 1.8.3 of the GDBM package is installed under the directory ‘/opt/gdbm-1.8.3’ the location of the header file would be,

/opt/gdbm-1.8.3/include/gdbm.h 

which is not part of the default gcc include path. Adding the appropriate directory to the include path with the command-line option -I allows the program to be compiled, but not linked:

$ gcc -Wall -I/opt/gdbm-1.8.3/include dbmain.c -lgdbm  /usr/bin/ld: cannot find -lgdbm collect2: ld returned 1 exit status 

The directory containing the library is still missing from the link path. It can be added to the link path using the following option:

-L/opt/gdbm-1.8.3/lib/ 

The following command line allows the program to be compiled and linked:

$ gcc -Wall -I/opt/gdbm-1.8.3/include     -L/opt/gdbm-1.8.3/lib dbmain.c -lgdbm 

This produces the final executable linked to the GDBM library. Before seeing how to run this executable we will take a brief look at the environment variables that affect the -I and -L options.

Note that you should never place the absolute paths of header files in #include statements in your source code, as this will prevent the program from compiling on other systems. The -I option or the INCLUDE_PATH variable described below should always be used to set the include path for header files.

Linking with external libraries

library is a collection of precompiled object files which can be linked into programs. The most common use of libraries is to provide system functions, such as the square root function sqrt found in the C math library.

Libraries are typically stored in special archive files with the extension ‘.a’, referred to as static libraries. They are created from object files with a separate tool, the GNU archiver ar, and used by the linker to resolve references to functions at compile-time. We will see later how to create libraries using the ar command (see section 10 Compiler-related tools). For simplicity, only static libraries are covered in this section--dynamic linking at runtime using shared libraries will be described in the next chapter.

The standard system libraries are usually found in the directories ‘/usr/lib’ and ‘/lib’.(5) For example, the C math library is typically stored in the file‘/usr/lib/libm.a’ on Unix-like systems. The corresponding prototype declarations for the functions in this library are given in the header file ‘/usr/include/math.h’. The C standard library itself is stored in ‘/usr/lib/libc.a’ and contains functions specified in the ANSI/ISO C standard, such as ‘printf’---this library is linked by default for every C program.

Here is an example program which makes a call to the external function sqrt in the math library ‘libm.a’:

#include  #include   int main (void) {   double x = sqrt (2.0);   printf ("The square root of 2.0 is %f\n", x);   return 0; } 

Trying to create an executable from this source file alone causes the compiler to give an error at the link stage:

$ gcc -Wall calc.c -o calc /tmp/ccbR6Ojm.o: In function `main': /tmp/ccbR6Ojm.o(.text+0x19): undefined reference    to `sqrt' 

The problem is that the reference to the sqrt function cannot be resolved without the external math library ‘libm.a’. The function sqrt is not defined in the program or the default library ‘libc.a’, and the compiler does not link to the file ‘libm.a’ unless it is explicitly selected. Incidentally, the file mentioned in the error message‘/tmp/ccbR60jm.o’ is a temporary object file created by the compiler from ‘calc.c’, in order to carry out the linking process.

To enable the compiler to link the sqrt function to the main program ‘calc.c’ we need to supply the library ‘libm.a’. One obvious but cumbersome way to do this is to specify it explicitly on the command line:

$ gcc -Wall calc.c /usr/lib/libm.a -o calc 

The library ‘libm.a’ contains object files for all the mathematical functions, such as sin, cos, exp, log and sqrt. The linker searches through these to find the object file containing the sqrt function.

Once the object file for the sqrt function has been found, the main program can be linked and a complete executable produced:

$ ./calc  The square root of 2.0 is 1.414214 

The executable file includes the machine code for the main function and the machine code for the sqrt function, copied from the corresponding object file in the library ‘libm.a’.

To avoid the need to specify long paths on the command line, the compiler provides a short-cut option ‘-l’ for linking against libraries. For example, the following command,

$ gcc -Wall calc.c -lm -o calc 

is equivalent to the original command above using the full library name ‘/usr/lib/libm.a’.

In general, the compiler option -lNAME will attempt to link object files with a library file ‘libNAME.a’ in the standard library directories. Additional directories can specified with command-line options and environment variables, to be discussed shortly. A large program will typically use many -l options to link libraries such as the math library, graphics libraries and networking libraries.

GCC-Using library header files

When using a library it is essential to include the appropriate header files, in order to declare the function arguments and return values with the correct types. Without declarations, the arguments of a function can be passed with the wrong type, causing corrupted results.

The following example shows another program which makes a function call to the C math library. In this case, the function pow is used to compute the cube of two (2 raised to the power of 3):

#include     int main (void) {   double x = pow (2.0, 3.0);   printf ("Two cubed is %f\n", x);   return 0; } 

However, the program contains an error--the #include statement for ‘math.h’ is missing, so the prototype double pow (double x, double y) given there will not be seen by the compiler.

Compiling the program without any warning options will produce an executable file which gives incorrect results:

$ gcc badpow.c -lm $ ./a.out Two cubed is 2.851120    (incorrect result, should be 8) 

The results are corrupted because the arguments and return value of the call to pow are passed with incorrect types.(6) This can be detected by turning on the warning option -Wall:

$ gcc -Wall badpow.c -lm badpow.c: In function `main': badpow.c:6: warning: implicit declaration of    function `pow' 

This example shows again the importance of using the warning option -Wall to detect serious problems that could otherwise easily be overlooked.

Thursday, August 20, 2009

How to Bulk Rename Files in Linux (Terminal or GUI)

If you have a directory of files that you would like to bulk rename, you can use the rename command from the terminal.

UPDATE: I believe the Perl-based rename command is only available on Debian-based Linux distros, but there are instructions on adding it to other distros below.

The syntax for the rename command is:

rename [ -v ] [ -n ] [ -f ] perlexpr [ files ]

-v means "verbose" and it will output the names of the files when it renames them. It is a good idea to use this feature so you can keep track of what is being renamed. It is also a good idea to do a test run with -n which will do a test run where it won't rename any files, but will show you a list of files that would be renamed.

The "perlexpr" part of the command is a Perl expression. Don't panic yet...

The "rename" command in action

Here is an example of the rename command:

rename -n ’s/\.htm$/\.html/’ *.htm

The -n means that it's a test run and will not actually change any files. It will show you a list of files that would be renamed if you removed the -n. In the case above, it will convert all files in the current directory from a file extension of .htm to .html.

If the output of the above test run looked ok then you could run the final version:

rename -v ’s/\.htm$/\.html/’ *.htm

The -v is optional, but it's a good idea to include it because it is the only record you will have of changes that were made by the rename command as shown in the sample output below:

$ rename -v 's/\.htm$/\.html/' *.htm 3.htm renamed as 3.html 4.htm renamed as 4.html 5.htm renamed as 5.html 

The tricky part in the middle is a Perl substitution with regular expressions, highlighted below:

rename -v ’s/\.htm$/\.html/’ *.htm

Tip: There is an intro to Perl regular expressions here.

Basically the "s" means substitute. The syntax is s/old/new/ — substitute the old with the new.

A . (period) has a special meaning in a regular expression — it means "match any character". We don't want to match any character in the example above. It should match only a period. The backslash is a way to "escape" the regular expression meaning of "any character" and just read it as a normal period.

The $ means the end of the string. \.htm$ means that it will match .htm but not .html.

It's fairly basic — substitute .htm with .html:

's/\.htm$/\.html/'

The last part of the command, highlighted below, means to apply the rename command to every file that ends with .htm (the * is a wildcard).

rename -v ’s/\.htm$/\.html/’ *.htm

Other Examples

Maybe you have a digital camera that takes photos with filenames something like 00001234.JPG, 00001235.JPG, 00001236.JPG. You could make the .JPG extension lowercase with the following command executed from the same directory as the images:

rename -v 's/\.JPG$/\.jpg/' *.JPG

Here is the output of the above command:

$ rename -v 's/\.JPG$/\.jpg/' *.JPG 00001111.JPG renamed as 00001111.jpg 00001112.JPG renamed as 00001112.jpg 00001113.JPG renamed as 00001113.jpg 

That is simple enough, as it is similar to the .html example earlier. You could also bulk rename them with something descriptive at the beginning like this:

Tip: Before trying more complicated renaming like in the example below, do a test run with the -n option as described at the beginning of this tutorial.

rename -v 's/(\d{8})\.JPG$/BeachPics_$1\.jpg/' *.JPG

That will change filenames that have the pattern ########.JPG (8 numbers and capital .JPG) to something like BeachPics_########.jpg (the same 8 numbers and changing the extension to lowercase .jpg). Here is a test run with the -n option:

$ rename -n 's/(\d{8})\.JPG$/BeachPics_$1\.jpg/' *.JPG 00001111.JPG renamed as BeachPics_00001111.jpg 00001112.JPG renamed as BeachPics_00001112.jpg 00001113.JPG renamed as BeachPics_00001113.jpg 

Here's a quick breakdown of the Perl substitution with the regular expression above.

The highlighted section below means to count 8 digits. The parentheses mean to save those 8 digits for later because they are going to be used again in the second half of the substitution:

's/(\d{8})\.JPG$/BeachPics_$1\.jpg/'

In the highlighted section below, it adds the string BeachPics, and underscore, and then the text in parentheses from the first half of the substitution. $1 will insert the string from the first set of parentheses that it finds — in this case the 8 digits. If you have more than one set of parentheses you can access the second set with the Perl variable $2 and so on.

's/(\d{8})\.JPG$/BeachPics_$1\.jpg/'

Final Refinement

The following variation would make even cleaner-looking filenames. See if you can figure out how it works:

$ rename -n 's/\d{5}(\d{3})\.JPG$/BeachPics_$1\.jpg/' *.JPG 00000123.JPG renamed as BeachPics_123.jpg 00000124.JPG renamed as BeachPics_124.jpg 00000125.JPG renamed as BeachPics_125.jpg