Thursday, August 14, 2014

Adding code snippets on blogger

A quick note on adding code snippets on blogger.
From the variety of tools available, I have found the best for formatting.

Modifying gcc optimizations while building nginx for debugging

Sometimes, during debugging I needed to disable optimizations. It took me a few searches to figure it out. So here is a short summary for anyone else who may face the same issue

Go to nginx/auto/cc/gcc file. Modify the CFLAGS to not include the variable NGX_GCC_OPT
In my (already modified Makefile) it was like this

which I changed to this

Tuesday, July 8, 2014



As the usage of nginx grows, people are enhancing its capabilities by adding a variety of modules. As the Emiller's guide to writing nginx modules states, nginx modules can take on 3 roles:

  1. handler, that handles the HTTP request
  2. filter, that handles the HTTP response and
  3. load balancer that choose which back-end server to send the HTTP request to.

In this article, we will discuss the order of execution of nginx filters.

Sometimes, when writing a new filter , you may wish to  look at the order of execution of module. For example, you may wish to always process decompressed response chunks.  For that, you need to be sure that the nginx gunzip body filter will decompress the response chunk before your module processes it.

Order of Execution of Nginx Modules

The order of filters is derived from the order of execution of nginx modules. The order of execution of nginx modules is implemented within the file auto/modules in the nginx source code.

 If you look inside the file, you will see that nginx divides the modules into various classes. For example, the statements

2      if [ $HTTP = YES ]; then
    modules="$modules $HTTP_MODULES $HTTP_FILTER_MODULES \
show  that
The modules which have the term FILTER in their names are the filter modules and this is what we discuss from now on.

For example, the gunzip filter module is classified as HTTP_FILTER while the Lua module is classified as HTTP_AUXILIARY_FILTERS.
Each of these class of filters is then invoked in a particular order as listed in the auto/modules code above.
Within each class, the filters are invoked in the order of their appearance in the file.

Now we need to figure out how the modules are invoked. The file auto/modules puts the list of modules in the $modules variable. The code at the bottom of this file (reproduced below) , then generates ngx_modules.c file. Nginx, then invokes these modules starting from the bottom in the ngx_modules.c file.

cat << END                                    > $NGX_MODULES_C
#include <ngx_config.h>
#include <ngx_core.h>


for mod in $modules
    echo "extern ngx_module_t  $mod;"         >> $NGX_MODULES_C
echo                                          >> $NGX_MODULES_C
echo 'ngx_module_t *ngx_modules[] = {'        >> $NGX_MODULES_C
for mod in $modules
    echo "    &$mod,"                         >> $NGX_MODULES_C
cat << END                                    >> $NGX_MODULES_C

Building call hierarchy for filter modules

Nginx builds  the call hierarchy for the filter modules at configuration time.

For this, it performs the following steps

  1. Initialise a global pointer to the top of the calling stack. Let’s call it ngx_ top_ filter
  2. Each filter module maintains a local variable that holds the the address of the next filter to be invoked. Let’s call this variable as ngx_next_filter 
  3.   At configuration time, nginx will invoke the init function for say module 1 (how the modules are going to be invoked will be explained shortly).

      1.  Module 1  will initialize ngx_top_filter = module1 and ngx_next_filter = NULL
TThe call hierarchy looks like this:

      2. then module 2 is invoked, it will initialize ngx_next_filter = ngx_top_filter (i.e. ngx_next_filter  will be initialized to module1 )  and ngx_top_filter = module2 i.e module 2 to place itself on top of the calling stack and module 1 after it
TThe call hierarchy looks like this:

   3. when module 3 is invoked, it will place itself on top of the calling stack with module 2 in the middle and module 1 at the bottom
TThe call hierarchy looks like this:

Thus  the module that is invoked first will end up at the bottom of the calling stack while the module that is invoked last will end up at the top.

Run time invocation

Once the calling stack has been built during the configuration time, the filters are expected to do the processing and invoked the next filter using the variable ngx_next_filter.

Additional implementation note

Nginx actually maintains two calling hierarchies. One, for the header filters and the other for the body filters.

Typically, a header filter will process the HTTP response headers while the body filter will process the HTTP body. Accordingly, the variables that you will see in the nginx source will be ngx_top_header_filter, ngx_next_header_filter for headers and ngx_top_body_filter, ngx_next_body_filter for body.

Though most of the filters that are available today stick to the aforementioned steps of invoking the next header filter , however, there is nothing binding a filter writer to follow this rule.

For example, a filter can choose not to invoke the next header filter in line under certain situations. This is, for example, done by modsecurity. Breaking the convention is usually not a good idea, but in case if you  are using a number of third-party filters and getting unexpected results, you may wish to check the calling hierarchy using a debugger or nginx logs.

Saturday, May 24, 2014

Restriction youtube for educational purposes

I wanted to give my phone to a young disadvantaged kid so that he can watch educational videos in his spare time from youtube. At the same time, I did not want him to be wasting time on internet or youtube.

So I used Kids video player to allow him to play only the videos that I load. This app uses another app called Kids place which is essentially a way to enable some apps to accessed only once the a password is added.
When working in tandem with Kids video player, Kids place will block the search youtube feature in kids video player until a password is added.

Thus, the kid will only be able to play the playlist that you want :-)

Friday, April 25, 2014

How to set up etags with emacs

  1. Create tags table:
    1. Navigate to the source code directory
    2. Run the command find . -name "*.[chCH]" -print | etags - (assuming that etags is already installed). This will create a TAGS file in the directory from which the aforementioned command is run
    3. In emacs do M-x visit-tags-table and navigate to the directory where the aforementioned command was run. This will load the TAGS table

Thursday, April 24, 2014

This summarizes the state of affairs in the past 2 weeks

Ubuntu: Updating repostiories

# make a backup of the repository
sudo cp /etc/apt/sources.list cp /etc/apt/sources.list.bkup

# manually edit the sources.list and add the repositories
sudo vi /etc/apt/sources.list

# update the repository information
sudo apt-get update

Saturday, January 18, 2014

Emacs configuration file

I’ve recently started using Emacs at work. I have used that at school earlier and then had to go to VI because none of the platforms that I worked on had enough space to support an Emacs editor. Emacs need some basic customisation before it can be used efficiently.

Here is a sample configuration file that I use. The comments provide explanation of the function of the various lines.

    ;;; Emacs Load Path
    (setq load-path (cons "~/emacs-config" load-path))
    ;;;(keyboard-translate ?\C-h ?\C-?)
    (global-set-key [(control h)] 'delete-backward-char)
    (global-set-key (kbd "C-^") 'help-command)
    (load "~/emacs-config/line-number")
    (load "~/emacs-config/whitespace-mode")
    (require 'whitespace)
    (setq whitespace-style '(face empty tabs lines-tail trailing))
    (global-whitespace-mode t)
    ;; Set emacs default C style to linux
    ;; Recommended
    (setq c-default-style "k&r")
    ;; Rationalize the undo operation in emacs
    (require 'redo+)
    (define-key global-map (kbd "C-q") 'undo)
    (define-key global-map (kbd "C-x C-q") 'redo)
    ;;; always print line numbers in the left margin
    (global-linum-mode t)
    ;;; Put backup files in a temp directory
    (setq backup-directory-alist
      `((".*" . ,temporary-file-directory)))
   (setq auto-save-file-name-transforms
      `((".*" ,temporary-file-directory t)))
    (show-paren-mode 1)
    ;;; Use spaces instead of tabs
    (setq-default indent-tabs-mode nil)
    ;; save desktop
    (desktop-save-mode 1)
    ;; speedbar in the same frame
    (require 'sr-speedbar)
    (load "~/emacs-config/sr-speedbar")
    ;; open speedbar on startup
    ;; Make speedbar list C functions
    (add-to-list 'speedbar-fetch-etags-parse-list
         '("\\.c" . speedbar-parse-c-or-c++tag))
    ;; windmove for easier switch between windows
    (when (fboundp 'windmove-default-keybindings)
    ;; emacs dev IDE
    (global-ede-mode 1)
    (require 'semantic/sb)
    (semantic-mode 1)
    ;;;; This snippet enables lua-mode
    ;; This line is not necessary, if lua-mode.el is already on your load-path
   (autoload 'lua-mode "lua-mode" "Lua editing mode." t)
   (add-to-list 'auto-mode-alist '("\\.lua$" . lua-mode))
   (add-to-list 'interpreter-mode-alist '("lua" . lua-mode))
    ;; set tab width
    (setq default-tab-width 4)
    (setq tab-stop-list (number-sequence 4 200 4))
    ;; Goto a particular line
    (global-set-key "\C-xg" 'goto-line)
     (add-to-list 'load-path "~/emacs-config/anything-config")
    (require 'anything-config)
    (setq anything-sources
       (list anything-c-source-buffers
     (setq frame-title-format '("USER1"))
     ;;(setq frame-title-format (list "%b - " (getenv "USERNAME") "@" (getenv "USERDOMAIN")))
    ;; prevent checking for files changed on disk because vmplayer does not sync
    (defun ask-user-about-supersession-threat (fn)
      "blatantly ignore files that changed on disk"
    (defun ask-user-about-lock (file opponent)
      "always grab lock"
 (global-set-key (kbd "<S-tab>") 'un-indent-by-removing-4-spaces)
 (defun un-indent-by-removing-4-spaces ()
  "remove 4 spaces from beginning of of line"
    ;; get rid of tabs at beginning of line
    (when (looking-at "^\\s-+")
     (untabify (match-beginning 0) (match-end 0)))
    (when (looking-at "^  ")
     (replace-match "")))))
  ;; custom-set-variables was added by Custom.
  ;; If you edit it by hand, you could mess it up, so be careful.
  ;; Your init file should contain only one such instance.
  ;; If there is more than one, they won't work right.
  '(column-number-mode t)
  '(show-paren-mode t)
  '(tool-bar-mode nil))
  ;; custom-set-faces was added by Custom.
  ;; If you edit it by hand, you could mess it up, so be careful.
  ;; Your init file should contain only one such instance.
  ;; If there is more than one, they won't work right.
  '(default ((t (:inherit nil :stipple nil :background "black" :foreground "white" :inverse-video nil :box nil :strike-through nil :overline nil :underline nil :slant normal :weight normal :height 98 :width normal :foundry "unknown" :family "DejaVu Sans Mono"))))

Monday, January 13, 2014

Using speech recognition (dragon naturally speaking) effectively

I work upwards of 10 hours a day on the computer and mobile phone. Needless to say, this is a potential to affect my health adversely. In order to reduce the strain on the fingers because of typing, I have been wanting to use speech recognition since about 2007. However, I could not use it as part of a daily routine because of the poor quality of speech recognition. 

I experimented with a variety of microphones, including those that were recommended by the vendors and popular sites such as this , but I would still not get an environment where I could just dictate sentences and not have to correct them after every few words. There was an additional problem of the program taking up a huge amount of memory. As a developer, I had to run other development packages on my computer simultaneously which caused the computer to slow down significantly.

A friend of mine told me that Dragon NaturallySpeaking version 12  has the ability of being able to dictate content wirelessly. This turned out to be a very useful feature. In fact, for me, it turned out to be a feature that helped me use voice recognition in my daily life instead of it just been a cool new software to try at times. I use the Dragon naturally speaking with the following arrangement:
  1. I have installed Dragon NaturallySpeaking on an Windows XP old laptop with 2G RAM. I connect to this wirelessly from  a dragon naturally speaking  app on my android mobile phone. Separately, on my main development machine, I login to the Windows XP laptop through remote desktop. I then dictate to Dragon NaturallySpeaking through the mobile phone and view the results on the Windows XP laptop using remote desktop.
  2. The superior quality of the microphone in the phone has dramatically help reduce voice recognition errors. Since the software is running on a separate machine, it does not slow down my main development machine.
Because of this feature, I have abandoned trying windows speech recognition, although I really liked it better while using browsers.