Standards

AutoKey Rocks

2012-01-22T12:04:00.001-08:00

Mac OS X has a built-in keyboard shortcut configuration tool in Preferences. You can set both global key bindings and application-specific key bindings.

I had used this to use Cmd key more often than Ctrl key. For example Google Chrome has default key mappings to go to the next tab by <ctrl-tab> and to the previous tab by <shift-ctrl-tab>. They break your left pinky easily. I mapped them ad <Cmd-j> and <Cmd-k>. That helped the finger.

But there is no such great built-in tool in the Linux world neither in Gnome nor KDE. There is a Keyboard Shortcuts in Preference of Gnome, but it's only for Gnome apps.

AutoKey

AutoKey is a similar tool to the Mac OS X built-in keybinding configuration tool. The differences are (1) AutoKey doesn't map a key to a function in the application's menu and (2) AutoKey enables you to run arbitrary scripts.

It's very easy to install on Debian family distributions such as Ubuntu. Gentoo, on the other hand, has an overlay that has autokey-gtk package, but it's broken. You have to get the source code and build it. It also requires you to install some dependencies manually, but it works.

autokey-gtk on gentoo

(Added on Jan 24 2012)

There is an issue in autokey that you cannot exclude a window easily. See the end of this discussion to solve it. http://groups.google.com/group/autokey-users/browse_thread/thread/658ad02cfbde8788

Compiling a Language to Whitespace

2011-12-26T01:14:00.001-08:00

Whitespace is a widely-known instruction set on a stack-machine virtual machine.

The below is a sample hello world in Whitespace intermediate language. I added some notes that begin with #.

Push 0
Push 72
Store
# storing the value 72 to 0th heap
Push 1
Push 101
Store
# storing the value 101 to 1st heap
Push 2
Push 108
Store
Push 3
Push 108
Store
Push 4
Push 111
Store
Push 5
Push 44
Store
Push 6
Push 32
Store
Push 7
Push 119
Store
Push 8
Push 111
Store
Push 9
Push 114
Store
Push 10
Push 108
Store
Push 11
Push 100
Store
Push 12
Push 32
Store
Push 13
Push 111
Store
Push 14
Push 102
Store
Push 15
Push 32
Store
Push 16
Push 115
Store
Push 17
Push 112
Store
Push 18
Push 97
Store
Push 19
Push 99
Store
Push 20
Push 101
Store
Push 21
Push 115
Store
# (cont.d...)

Push 22
Push 33
Store
Push 23
Push 0
Store
Push 0
Call "\t \t  \t\t   \t \t\t\t \t  \t \t\t  \t  \t\t\t \t\t\t \t\t\t "
# Jumping to the ☃ mark with remembering this place to return later
Call "\t \t  \t\t  \t\t\t \t\t \t  \t \t\t   \t\t \t\t \t\t\t \t\t\t \t \t  \t\t  \t\t\t \t\t "
# Jumping to the ☁ mark with remembering this place to return later
End
# Terminating program.
Label "  \t  \t\t   \t  \t\t \t    \t\t "
Infix Plus
Return
Label "\t \t  \t\t   \t \t\t\t \t  \t \t\t  \t  \t\t\t \t\t\t \t\t\t "
# ☃
Dup
# Copying the top value of the stack. It's 0 (if you see here for the first time.)
Retrieve
# Getting the 0th value of the heap. It's 72 (if you see here for the first time.)
Dup
If Zero "  \t  \t\t  \t\t\t \t\t \t \t  \t\t \t\t\t\t\t \t \t \t  \t\t   \t \t\t\t \t  \t \t\t  \t  \t\t\t \t\t\t \t\t\t "
OutputChar
Push 1
Infix Plus
Jump "\t \t  \t\t   \t \t\t\t \t  \t \t\t  \t  \t\t\t \t\t\t \t\t\t "
Label "  \t  \t\t  \t\t\t \t\t \t \t  \t\t \t\t\t\t\t \t \t \t  \t\t   \t \t\t\t \t  \t \t\t  \t  \t\t\t \t\t\t \t\t\t "
Discard
Discard
Return
# Ending the function call and going back to the previous place.
Label "  \t  \t\t \t    \t\t \t \t  \t\t  \t  \t\t\t "
Dup
Dup
ReadChar
Retrieve
Dup
Push 10
Infix Minus
If Zero "  \t  \t\t  \t\t\t \t\t \t \t  \t\t \t\t\t\t\t \t   \t  \t\t \t    \t\t \t \t  \t\t  \t  \t\t\t "
Discard
Push 1
Infix Plus
Jump "  \t  \t\t \t    \t\t \t \t  \t\t  \t  \t\t\t "
Label "  \t  \t\t  \t\t\t \t\t \t \t  \t\t \t\t\t\t\t \t   \t  \t\t \t    \t\t \t \t  \t\t  \t  \t\t\t "
Discard
Push 1
Infix Plus
Push 0
Store
Return
Label "\t \t  \t\t  \t\t\t \t\t \t  \t \t\t   \t\t \t\t \t\t\t \t\t\t \t \t  \t\t  \t\t\t \t\t "
# ☁
Push 10
Push 13
OutputChar
OutputChar
Return

And the result is

Hello, world of spaces!

You can run it with a little bit fixes of the official whitespace implementation wspace 0.3 written in Haskell.

A Language

I made an experimental small language and it's compiler.

(begin
  (def 0 98)
  (f 90 7)
  (putc (ref 0))
  (end)
  (defn f (x y)
        (putc (g x y)))
  (defn g (x y)
        (+ x y)))

Syntax: an expression begins with "(" and a name, arguments for it, and end with ")". If an expression has a value, you have to use the value with enclosing another expression. Otherwise the behavior is undefined. If an expression does not have a value, you must not enclose it as an argument. Otherwise the behavior is undefined.

begin
- combines some expressions that don't have return values
def {index} {value}
- assigns the {value} to {index}th slot of heap
ref {index} -> {value}
- returns the {value} of {index}th slot of heap
putc {value}
- outputs a character which ASCII code is {value}
end
- terminates program
defn {name} ({args}) {body}
- defines a function
- if a program come to defn without using call, the behavior is undefined.

* + {value} {value} -> {value} * obviously

You can "call" a function you made just with ({name} {arg1} {arg2}). You can use arguments of the function just by an identifier like x.

Sample code

(begin
  (putc (+ 50 40))
  (end))

That is compiled to

Push 50
Push 40
Infix Plus
OutputChar
End

and shows 'Z'.

I little bit complicated example

(begin
  (def 0 98)
  (f 90 7)
  (putc (ref 0))
  (end)
  (defn f (x y)
        (putc (g x y)))
  (defn g (x y)
        (+ x y)))

compiled to

Push 0
Push 98
Store
Push 90
Push 7
Call "f"
Push 0
Retrieve
OutputChar
End
Label "f"
Push (-1)
Swap
Store
Push (-2)
Swap
Store
Push (-1)
Retrieve
Push (-2)
Retrieve
Call "g"
OutputChar
Return
Label "g"
Push (-3)
Swap
Store
Push (-4)
Swap
Store
Push (-3)
Retrieve
Push (-4)
Retrieve
Infix Plus
Return

and shows "ab".

You may have noticed that the function argument is in negative number of heaps. If you updates them like by (def -3 100) it may result in breaking something, but since this implementation doesn't support negative literals, it remains safe.

The compiler is below, written in Haskell.

import qualified VM as V
import qualified Text.Parsec as P
import Control.Applicative ((<|>), (<$>))
import qualified Control.Monad.State as S
import qualified Data.Map as M
import Data.Maybe (fromJust)

data Intermediate = Comment String
  | Inst V.Instruction
  | Paramdef String
  | Paramref String
  deriving Show

type LispParser = P.ParsecT String () (S.State String)
type ParamMap = M.Map String Integer

main :: IO ()
main = do
  code <- readFile "hworld.lisp"
  --mapM_ print $ parse code
  let runtime = compile (parse code)
  mapM_ print runtime
  putStrLn "--"
  V.vm (V.VM runtime (V.Stack []) (V.Stack []) M.empty 0)

parse :: String -> [Intermediate]
parse str = either (error . show) id $
  S.evalState (P.runPT parseExpr () "lisp" str) "toplevel"

parseExpr :: LispParser [Intermediate]
parseExpr = P.try parseInt
         <|> parseDefn
         <|> parseBuiltin
         <|> parseApply
         <|> parseVar

parseInt :: LispParser [Intermediate]
parseInt = do
  x <- P.many1 P.digit
  return [Inst $ V.Push $ read x]

parseAtom :: LispParser String
parseAtom = P.many1 $ P.noneOf " \t\n()"

parseDefn :: LispParser [Intermediate]
parseDefn = do
  P.try $ do
    ignoringSpaces $ P.char '('
    ignoringSpaces $ P.string "defn"
  fname <- requireSpaces parseAtom
  S.lift $ S.put fname

  ignoringSpaces $ P.char '('
  names <- ignoringSpaces $ parseAtom `P.sepBy` P.skipMany1 P.space
  ignoringSpaces $ P.char ')'

  body <- ignoringSpaces parseExpr
  ignoringSpaces $ P.char ')'
  S.lift $ S.put "toplevel"
  return $
    Comment "(defn" :
    Inst (V.Label fname) :
    map (Paramdef . ((fname ++ "/") ++)) names ++
    body ++ [Inst V.Return] ++ [Comment ")"]

parseBuiltin :: LispParser [Intermediate]
parseBuiltin = P.try $ do
  (fname, xs) <- atomAndArgs
  x <- case (fname, length xs) of
       ("+", 2) -> return [Inst $ V.Infix V.Plus]
       ("putc", 1) -> return [Inst V.OutputChar]
       ("def", 2) -> return [Inst V.Store]
       ("ref", 1) -> return [Inst V.Retrieve]
       ("end", 0) -> return [Inst V.End]
       ("begin", _) -> return []
       _ -> fail "omg"
  return $ Comment ('(' : fname) : concat xs ++ x ++ [Comment ")"]

parseApply :: LispParser [Intermediate]
parseApply = do
  (fname, xs) <- atomAndArgs
  return $ concat xs ++ [Inst $ V.Call fname]

atomAndArgs :: LispParser (String, [[Intermediate]])
atomAndArgs = do
  ignoringSpaces $ P.char '('
  fname <- ignoringSpaces parseAtom
  xs <- ignoringSpaces $ parseExpr `P.sepBy` P.many1 P.space
  P.char ')'
  return (fname, xs)

parseVar :: LispParser [Intermediate]
parseVar = do
  name <- ignoringSpaces $ P.many1 $ P.noneOf " \t\n()"
  fname <- S.lift S.get
  return [Paramref $ fname ++ '/' : name]

ignoringSpaces :: LispParser a -> LispParser a
ignoringSpaces f = P.skipMany P.space >> f

requireSpaces :: LispParser a -> LispParser a
requireSpaces f = P.skipMany1 P.space >> f

compile :: [Intermediate] -> [V.Instruction]
compile inters = concat $ S.evalState (mapM compile' inters) M.empty

compile' :: Intermediate -> S.State ParamMap [V.Instruction]
compile' (Comment _) = return []
compile' (Inst x) = return [x]
compile' (Paramdef name) = do
  idx <- pred . negate . fromIntegral . M.size <$> S.get
  S.modify $ M.insert name idx
  return [V.Push idx, V.Swap, V.Store]
compile' (Paramref name) = do
  idx <- fromJust . M.lookup name <$> S.get
  return [V.Push idx, V.Retrieve]

This code depends on VM.hs from wspace-0.3 to share the data structure of VM Instruction and to execute the compiled program. If you only want to compile given programs, you don't need VM.hs but just to add the following definition.

data Instruction =
       Push Integer
     | Dup
     | Ref Int
     | Slide Int
     | Swap
     | Discard
     | Infix Op
     | Store
     | Retrieve
     | Label Label
     | Call Label
     | Jump Label
     | If Test Label
     | Return
     | OutputChar
     | OutputNum
     | ReadChar
     | ReadNum
     | End
   deriving (Show,Eq)

By the way wspace-0.3 had an issue that it can only handle sequential indices of heap. You can store values in 0th, 1st and 2nd slots of heap, but you cannot store in 100th without completing all indices between 0 to 100. I wrote a patch to allow any index. Feel free to use it.

diff --git VM.hs VM.hs
index c9e96ab..bb74374 100644
--- VM.hs
+++ VM.hs
@@ -1,6 +1,8 @@
 module VM where

 import IO
+import qualified Data.Map as M
+import Data.Maybe (fromJust)

 {- Stack machine for running whitespace programs -}

@@ -35,7 +37,7 @@ type Loc = Integer

 type Program = [Instruction]
 newtype Stack = Stack [Integer]
-type Heap = [Integer]
+type Heap = M.Map Integer Integer

 data VMState = VM {
         program :: Program,
@@ -130,13 +132,7 @@ findLabel' m (_:xs) i = findLabel' m xs (i+1)
 -- Heap management

 retrieve :: Integer -> Heap -> IO Integer
-retrieve x heap = return (heap!!(fromInteger x))
+retrieve x heap = return $ fromJust $ M.lookup x heap

 store :: Integer -> Integer -> Heap -> IO Heap
-store x 0 (h:hs) = return (x:hs)
-store x n (h:hs) = do hp <- store x (n-1) hs
-             return (h:hp)
-store x 0 [] = return (x:[])
-store x n [] = do hp <- store x (n-1) [] 
-         return (0:hp)
-
+store x n h = return $ M.insert n x h

Continuous-Passing Conversion in Haskell

2011-12-22T22:13:00.001-08:00

http://en.wikipedia.org/wiki/Continuation-passing_style

Convert from

(+ (f 0 (g 1)) 2)

(g' (lambda (r0) (f' (lambda (r1) (+ r1 2)) 0 r0)) 1)

where data structure internally in Haskell is like

data AST = Node [AST] | Leaf Value
data Value = IntVal Int | Plus | Atom String | Lambda [String]

Implementation and description

import qualified Control.Monad.State as S

data AST = Node [AST] | Leaf Value
instance Show AST where
  show (Node xs) = "(" ++ unwords (map show xs) ++ ")"
  show (Leaf v) = show v

data Value = IntVal Int | Plus | Atom String | Lambda [String]
instance Show Value where
  show (IntVal i) = show i
  show Plus = "+"
  show (Atom name) = name
  show (Lambda names) = "lambda (" ++ unwords names ++ ")"

-- (+ (f 0 (g 1)) 2)
-- (g' (lambda (r0) (f' (lambda (r1) (+ r1 2)) 0 r0)) 1)
program :: AST
program = Node [Leaf Plus,
  Node [Leaf (Atom "f"), Leaf (IntVal 0), Node [Leaf (Atom "g"), Leaf (IntVal 1)]],
  Leaf (IntVal 2)]

main = do
  print program
  print $ cps program

cps :: AST -> AST
cps ast =
  let (newAst, modifiers) = S.runState (cps' ast) [] in
      foldl (flip ($)) newAst modifiers

cps' :: AST -> S.State [AST -> AST] AST
cps' (Node (Leaf (Atom f) : xs)) = do
  xs' <- mapM cps' xs
  n <- length `fmap` S.get
  let name = 'r' : show n
  append $ \root -> Node $
    (Leaf . Atom $ f ++ "'") :
    Node [Leaf (Lambda [name]), root] :
    xs'
  return $ Leaf (Atom name)
cps' (Node xs) = Node `fmap` mapM cps' xs
cps' c@(Leaf _) = return c

append x = S.modify (x :)

This converts correctly.

I used State Monad to modify given tree. The function cps starts state and the actual function cps' traverses given AST subtrees recursively.

(+ (f 0 (g 1)) 2)
   ^^^^^^^^^^^

When cps' sees this subtree, oh yes the first item of the list is a user-defined function and it's not tail-call, so cps' wants to replace the part with a new variable (say r), and enclose whole tree with new function f' and the arguments.

(f' (lambda (r) ((+ r 2) 0 (g 1))))
^^^^^^^^^^^^^^^^^   ^   ^^^^^^^^^^^

It's easy to change subtree but it's not trivial to change outside the subtree. But fortunately we already know that we only have to enclose something around the whole tree, so you can just save a function in state.

After cps' process is done, you apply all functions that the state has accumulatively to enclose trees. That will complete the job.

Unlambda Interpreter in Haskell

2011-12-20T20:56:00.001-08:00

Unlambda is a minimal, "nearly pure"[1] functional programming language invented by David Madore. It is based on combinatory logic, a version of the lambda calculus that omits the lambda operator. It relies mainly on two built-in functions (s and k) and an "apply" operator (written `, the backquote character). These alone make it Turing-complete, but there are also some I/O functions to make it possible to interact with the user, some shortcut functions and a function for lazy evaluation. There are no variables in the language.

http://en.wikipedia.org/wiki/Unlambda

import qualified Text.Parsec as P
import Control.Applicative ((*>), (<$>), (<*>))

data AST = Apply AST AST | Val Value
instance Show AST where
  show (Apply a b) = "(" ++ show a ++ " " ++ show b ++ ")"
  show (Val (Dot c)) = "put-" ++ [c]
  show (Val (Builtin c)) = [c]

data Value = Dot Char
  | Builtin Char
  | PendingK Value
  | PendingS1 Value
  | PendingS2 Value Value
  deriving Show

main = do
  let helloworld = "`r```````````.H.e.l.l.o. .w.o.r.l.di"
  let fibonacci = "```s``s``sii`ki`k.*``s``s`ks``s`k`s`ks``s``s`ks``s`k`s`kr``s`k`sikk`k``s`ksk"
  print $ desugar $ parse helloworld
  eval $ desugar $ parse helloworld
  --eval $ desugar $ parse fibonacci

parse :: String -> AST
parse = either (error . show) id . P.parse parse' "unlambda"

parse' = P.try (P.char '`' *> (Apply <$> parse' <*> parse'))
         P.<|>
         P.try (P.char '.' *> (Val . Dot <$> P.anyChar))
         P.<|>
         P.try (Val . Builtin <$> P.anyChar)

desugar :: AST -> AST
desugar (Apply a b) = Apply (desugar a) (desugar b)
desugar (Val (Builtin 'r')) = Val (Dot '\n')
desugar (Val (Builtin 'i')) = Apply (Apply (Val (Builtin 's')) (Val (Builtin 'k'))) (Val (Builtin 'k')) -- i = ``skk
desugar x = x

eval :: AST -> IO (Value)
eval (Apply a b) = do
  a' <- eval a
  b' <- eval b
  apply a' b'
eval (Val x) = return x

apply :: Value -> Value -> IO Value
apply (Dot c) x = putChar c >> return x
apply (Builtin 'k') x = return $ PendingK x
apply (Builtin 's') x = return $ PendingS1 x
apply (PendingK x) y = return $ x
apply (PendingS1 x) y = return $ PendingS2 x y
apply (PendingS2 x y) z = do
  a <- apply x z
  b <- apply y z
  apply a b

parse the given string to abstract syntax tree
desugar the ast; expanding macros like r or i.
interpreter evaluates all nodes!

AST

(put-\n (((((((((((put-H put-e) put-l) put-l) put-o) put- ) put-w) put-o) put-r) put-l) put-d) ((s k) k)))

Result of helloworld

Hello world

Result of fibonacci

*
*
**
***
*****
********
*************
*********************
**********************************
*******************************************************
*****************************************************************************************

(added at Tue Dec 20 23:57:12 PST 2011)

I also made a stackmachine-based virtual machine and a compiler for it.

https://gist.github.com/1505131

This was actually much simpler/easier than I thought. There's a difference between pure interpreter and this virtualmachine, but it's not very big.

For example very short program "hi" that shows "hi" is "``.h.ii" in unlambda. First this parser converts the text to AST.

((put-h put-i) ((s k) k))

Then the compiler converts the tree to sequence of instructions.

IPush (Dot 'h')
IPush (Dot 'i')
IApply
IPush (Builtin 's')
IPush (Builtin 'k')
IApply
IPush (Builtin 'k')
IApply
IApply

Then the virtualmachine runtime will run it.

hi

Read Raw POST Request Body in Compojure

2011-12-18T18:29:00.001-08:00

Brief introduction to Compojure

Compojure gives you a shortcut to make a monofunctional web application. For example a Lingr bot is a web application that only need to be responsible with one single endpoint that handles POST request. The below is a web application that only shows "hello" in / endpoint with GET request.

(defroutes hello
           (GET "/" [] "hello"))
(run-jetty hello {:port 80})

Note that it requires you to be the root of the system if you are going to run a web app on port 80.

The main part of the app is just 3 lines of code. That reminds me of code examples for Sinatra, a Ruby web library.

get '/' do
  'hello'
end

Anyways the Compojure example code doesn't work only with the main logic. You are supposed to make a Leiningen project usually to manage the app and its dependent libraries.

$ lein new hello
$ cd hello

project.clj

(defproject hello "1.0.0-SNAPSHOT"
            :main hello.core
            :description "FIXME: write description"
            :dependencies [[org.clojure/clojure "1.2.1"]
                           [compojure "1.0.0-RC2"]
                           [ring "1.0.1"]])

src/hello/core.clj

(ns hello.core
  (:use
    compojure.core
    ring.adapter.jetty))

(defroutes hello
           (GET "/" [] "hello"))
(run-jetty hello {:port 80})

then

$ lein deps
$ lein run

it should work.

Parameters

(defroutes hello
           (GET "/" [] "hello"))
(run-jetty hello {:port 80})

The 2nd argument of GET, [] in this case, is parameter list for the expression you give in 3rd argument, which mostly for referring GET parameters. That's actually a hashmap that contains :params key which value is also a hashmap of GET parameters. Ditto in POST.

How can we get the raw post parameter?

(POST "/" {params :params} (...))

In that way you cannot get raw data because it's after the process. You can reconstruct the raw data only when the given parameter is like proper a=1\nb=2 form. These days some web apis are required to handle raw POST data, which is mostly in JSON, like a Lingr Bot API.

The answer is in :body of the parameter, but it's not a String but a mysterious HttpParser.Input object, assuming you are using ring as the middleware.

http://jetty.codehaus.org/jetty/jetty-6/apidocs/org/mortbay/jetty/HttpParser.Input.html

This class looks weird because even though this has read() method the return value type isn't String but int. The other read() looks like you are supposed to pass a mutable data and refer the changed data.

Fortunately we can use slurp Clojure function to hide this complicated behaviour.

(defroutes hello
           (POST "/" {body :body} (slurp body)))

This shows the given raw POST parameter!

Lazy List in C

2011-11-15T15:43:00.001-08:00

#include <stdio.h>
#include <stdlib.h>

typedef struct list_ {
  int x;
  struct closure_int_int_list_ *tail;
} *list;

typedef struct closure_int_int_list_ {
  list (*call)(int, int);
  int x1;
  int x2;
} *closure_int_int_list;

closure_int_int_list newclosure(list call(int, int), int x1, int x2)
{
  closure_int_int_list c;
  c = (closure_int_int_list)malloc(sizeof(*c));
  c->call = call;
  c->x1 = x1;
  c->x2 = x2;
  return c;
}

list newlist(int x, closure_int_int_list tail)
{
  list xs = (list)malloc(sizeof(struct list_));
  xs->x = x;
  xs->tail = tail;
  return xs;
}

list listtail(list xs)
{
  if (xs->tail == NULL) return NULL;
  return xs->tail->call(xs->tail->x1, xs->tail->x2);
}

void deletelist(list xs)
{
  free(xs->tail);
  free(xs);
}

int *takelist(int num, list xs)
{
  int *array;
  int i;
  list p;
  array = (int *)malloc(sizeof(int) * num);
  p = xs;
  for (i = 0; i < num; ++i) {
    array[i] = p->x;
    p = listtail(p);
  }
  return array;
}

list fibnext(int a, int b)
{
  return newlist(b, newclosure(fibnext, b, a + b));
}

void printarray(int *xs, int size)
{
  int i;
  for (i = 0; i < size; ++i) {
    printf("%d ", xs[i]);
  }
}

int main(int argc, char const* argv[])
{
  list xs;
  int *array;
  xs = newlist(1, newclosure(fibnext, 1, 1));
  array = takelist(10, xs);
  printarray(array, 10);
  free(array);
  deletelist(xs);
  return 0;
}

result:

1 1 2 3 5 8 13 21 34 55

Type Inferences of Ambiguous Literals

2011-11-13T20:53:00.001-08:00

The Haskell code below works.

main = print $ x + y
x = 1
y = 2.3

This results 3.3. x isn't Int because x is used with (+) operator that also takes 2.3.

On the other hand, the code below causes a type error in compile time.

main = print $ x + y
x = 1
y = 2.3

z = x :: Int

error:

No instance for (Fractional Int)
  arising from the literal `2.3'
Possible fix: add an instance declaration for (Fractional Int)
In the expression: 2.3
In an equation for `y': y = 2.3

You can make x ambiguous with NoMonomorphismRestriction option.

{-# LANGUAGE NoMonomorphismRestriction #-}

or -XNoMonomorphismRestriction in command line option.

http://stackoverflow.com/questions/7055146/why-do-3-and-x-which-was-assigned-3-have-different-inferred-types-in-haskell

Thanks @ikegami__!

Lossless Data Compressions

2011-11-03T03:25:00.001-07:00

Run-length encoding

http://en.wikipedia.org/wiki/Run-length_encoding

One of the easiest encoding method to write.

An implementatin in Haskell:

import Data.List (group)
main = do
  print $ runlength "AAAAABBCBADDDDDDDD"

runlength = concatMap f . group
  where
    f xs@(x:_) = x : show (length xs)

"A5B2C1B1A1D8"

Huffman coding

http://en.wikipedia.org/wiki/Huffman_coding

An implementatin in Haskell:

import Data.List (group, sort, sortBy)
import Data.Function (on)
import Data.Map (fromList, lookup)
import Data.Maybe (fromJust)
import Prelude hiding (lookup)

main = print $ huffman "AAAAABBCBADDDDDDDD"

huffman s = flip concatMap s $ fromJust . flip lookup (huffmanMap s)

huffmanMap s =
  let x = map head . sortBy (compare `on` length) . group . sort $ s in
      fromList $ zipWith ((,)) x (bits $ length x)
bits :: Int -> [[Int]]
bits length = (take length $ repeat 1) : reverse (take (length - 1) $ iterate (1 :) [0])

[1,0,1,0,1,0,1,0,1,0,1,1,0,1,1,0,1,1,1,1,1,1,0,1,0,0,0,0,0,0,0,0,0]

BPE: Byte Pair Encoding

http://en.wikipedia.org/wiki/Byte_pair_encoding

import Data.List (group, sort, sortBy, (\\))
import Data.Function (on)
import qualified Data.Map as M
import Data.Map (insert, empty, notMember)
import Data.Maybe (fromJust)
import Safe (fromJustDef)

main = do
  let orig = "ABCDCDABCDCDE"
  print orig
  let (encoded, table) = bpe orig
  print (encoded, table)
  print $ bpeDecode encoded table

bpe xs = bpe' xs empty
bpe' xs table =
  let x = head $ sortBy (flip compare `on` length) $ group $ sort $ zipWith ((,)) xs (tail xs) in
      if length x == 1
         then (xs, table)
         else let (a, b) = head x
                  new = pickupOther xs table in
                  bpe' (replace2 xs a b new) (insert new (a, b) table)

bpeDecode xs table = concatMap (replace (expand (M.map (\(a, b) -> [a, b]) table))) xs
  where
    replace :: M.Map Char String -> Char -> String
    replace expandedTable c = maybe [c] id $ M.lookup c expandedTable
    expand :: M.Map Char String -> M.Map Char String
    expand table = M.map (concatMap f) table
      where
        f :: Char -> String
        f c = maybe [c] (concatMap f) $ M.lookup c table

pickupOther xs table =
  head $ filter (flip notMember table) $ ['Z', 'Y'..'A'] \\ xs

replace2 (a:[]) _ _ _ = [a]
replace2 (a:b:xs) c d e
  | a == c && b == d = e : replace2 xs c d e
  | otherwise = a : replace2 (b : xs) c d e

"ABCDCDABCDCDE"
("WWE",fromList [('W',('X','Z')),('X',('Y','Z')),('Y',('A','B')),('Z',('C','D'))])
"ABCDCDABCDCDE"

Optimizer Comparisons -- Skip Nops

2011-10-31T22:44:00.001-07:00

In Haskell

The following Haskell code is supposed to just output "hello".

main = print $ times 1000000000000 id "hello"

times 0 _ x = x
times n f x = times (n-1) f (f x)

The times function I defined applies the given function many times.

times 3 f x is...
- times 2 f (f x)
- times 1 f (f (f x))
- times 0 f (f (f (f x)))
- f (f (f x))

id, defined in Prelude, just returns the given argument. You may want to assume the GHC internal optimizer to eliminate any id call. In other words, the Haskell code in the very beginning is supposed to be transformed just to main = print "hello".

Unfortunately, no it doesn't, even with -O2 option.

$ ghc -O2 a.hs -o a
$ ./a
... (long long hours) ..

In C

I just translated the original Haskell code to C.

#include "stdio.h"

char *id(char *x) {
  return x;
}
char *times(long long n, char *(*f)(char *), char *x) {
  if (n == 0) {
    return x;
  } else {
    return times(n-1, f, f(x));
  }
}

int main(int argc, char const* argv[]) {
  puts(times(1000000000000LL, id, "hello"));
  return 0;
}

I tried it with GCC and Clang.

GCC on Mac:

i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
```
$ gcc -O2 a.c -o a
$ ./a
... (long long hours) ..
```

The assembly code for the main function with -O2 -m32 option is the following.

_main:
  pushl %ebp
  movl  %esp, %ebp
  pushl %ebx
  subl  $20, %esp
  call  L13
"L00000000001$pb":
L13:
  popl  %ebx
  leal  LC0-"L00000000001$pb"(%ebx), %eax
  movl  %eax, 8(%esp)
  leal  _id-"L00000000001$pb"(%ebx), %eax
  movl  %eax, 4(%esp)
  movl  $-727379968, (%esp)
  call  _times
  movl  %eax, (%esp)
  call  _puts
  xorl  %eax, %eax
  addl  $20, %esp
  popl  %ebx
  leave
  ret

You see it's calling _times for sure.

Clang:

$ clang -O2 a.c -o a
$ ./a
hello

This gives you the message immediately!

The below is the generated LLVM IR. You can easily tell that it just calls puts function directly. Note that the @.str const is just "hello" ending with null.

define i32 @main(i32 %argc, i8** nocapture %argv) nounwind uwtable ssp {
entry:
  %call1 = tail call i32 @puts(i8* getelementptr inbounds ([6 x i8]* @.str, i64 0, i64 0)) nounwind
  ret i32 0
}

GCC on Linux:

gcc 4.4

main:
  pushl %ebp
  movl  %esp, %ebp
  andl  $-16, %esp
  subl  $16, %esp
  movl  $.LC0, (%esp)
  call  puts
  xorl  %eax, %eax
  leave
  ret

It ran like the one with Clang!

When the big number isn't long long

I wrote the first argument of times in C as int at first.

I got warning messages from both GCC and Clang that the value, 1000000000000, was too big for int. In the case, GCC on Linux made the following assembly code.

main:
  pushl %ebp
  movl  %esp, %ebp
.L11:
  jmp   .L11

It's just an infinite loop! That actually makes sense because any int values never go to non-int value 1000000000000.

Summary

GHC doesn't eliminate id in optimization
GCC eliminates it except for Apple GCC 4.2
Clang eliminates id
Some optimizations can conflict if the code depends on undefined behaviour

How To Compile C to IA32

2011-10-30T10:33:00.001-07:00

Q. How to compile a C file to IA32 assembly language on AMD64 Debian?

$ gcc -S -m32 a.c
In file included from /usr/include/features.h:378,
                 from /usr/include/stdio.h:28,
                 from a.c:1:
/usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No such file or directory

A. install gcc-multilib.

$ sudo aptitude install gcc-multilib

URL Encodings

2011-10-28T18:20:00.001-07:00

URL Encoding, specifically Percent-encoding, is one of common encodings that you may use in arbitorary programming languages.

Let me encode a text into URL Encoding, which includes UTF-8 Japanese characters.

JavaScript (node.js)

Use encodeURIComponent.

var sys = require('sys');
sys.puts(encodeURIComponent('abc あいう-#!@'));

result:

abc%20%E3%81%82%E3%81%84%E3%81%86-%23!%40

Note that escape and encodeURI aren't proper for URI encoding.

sys.puts(escape('abc あいう-#!@'));
//=> abc%20%u3042%u3044%u3046-%23%21@
sys.puts(encodeURI('abc あいう-#!@'));
//=> abc%20%E3%81%82%E3%81%84%E3%81%86-#!@
sys.puts(encodeURIComponent('abc あいう-#!@'));
//=> abc%20%E3%81%82%E3%81%84%E3%81%86-%23!%40

(Thanks @javascripter!)

Ruby

Use ERB::Util.url_encode.

# coding: utf-8
require 'erb'
puts ERB::Util.url_encode 'abc あいう-#!@'

result:

abc%20%E3%81%82%E3%81%84%E3%81%86-%23%21%40

This result is exactly same to JavaScript's except for "!".

Note that URI.encode or URI.encode_www_form_component aren't proper.

require 'uri'
puts URI.encode 'abc あいう-#!@'
#=> abc%20%E3%81%82%E3%81%84%E3%81%86-%23!@
puts URI.encode_www_form_component 'abc あいう-#!@'
#=> abc+%E3%81%82%E3%81%84%E3%81%86-%23%21%40

Python

Use urllib.quote.

# coding: utf-8
import urllib
print urllib.quote('abc あいう-#!@')

result:

abc%20%E3%81%82%E3%81%84%E3%81%86-%23%21%40

(Thanks @raa0121!)

Vim script

Use Vital.Web.Http.encodeURI.

let H = vital#of('vital').import('Web.Http')
echo H.encodeURI('abc あいう-#!@')

result:

abc%20%E3%81%82%E3%81%84%E3%81%86-%23%21%40

Haskell

Use Network.HTTP.urlEncode. But...

import Network.HTTP (urlEncode)
main = putStrLn $ urlEncode "abc あいう-#!@"

result:

abc%20%30%42%30%44%30%46-%23%21%40

oh...

Converting From/To String

2011-10-25T23:27:00.001-07:00

Haskell has three dominant types of the implementations of strings; String, ByteString and Text(*1). Since String is the default literal of strings, most libraries only support String as strings, even though you can write ByteString or Text directly in a Haskell code with OverloadedStrings pragma.

Haskell has IsString class that means you can convert the type to String, using fromString function. IsString only requires the type to have the fromString function.

-- in Data.String
class IsString a where
  fromString :: String -> a

ToString

I implemented ToString class that you can convert from the type to String by toString function, oppositely to the IsString class and its fromString function.

{-# LANGUAGE TypeSynonymInstances #-}
import qualified Data.Text as T
import qualified Data.ByteString.Char8 as C
import qualified Data.ByteString as B

class ToString a where
  toString :: a -> String

instance ToString String where
  toString = id

instance ToString T.Text where
  toString = T.unpack

instance ToString B.ByteString where
  toString = C.unpack

You can use them like the following way.

{-# LANGUAGE OverloadedStrings #-}
main = do
  putStrLn $ toString ("hello world!" :: String)
  putStrLn $ toString ("hello world!" :: T.Text)
  putStrLn $ toString ("hello world!" :: B.ByteString)

It might remind you about show :: a -> String. show and the toString I defined here have the same type but the purpose and the behaviour are different. show is to give human-readable and machine-parsable String represantation while toString is just to convert from a type which is an implementation of the concept of strings to String.

If you are familiar with Ruby, this may make sense; show in Haskell is inspect in Ruby while toString in Haskell is to_str in Ruby. Note that that's not to_s whom Integer or some other non-string classes also have.

Use case

An example is Network.HTTP.urlEncode function; the function is String -> String but you may want to use it for Text values.

import qualified Data.Text as T
import qualified Network.HTTP as H
urlEncodeText :: T.Text -> T.Text
urlEncodeText = T.pack . H.urlEncode . T.unpack

You can define Text-specific version of urlEncode easily. You can define ByteString-specific version as well in the same way. But you cannot have general one.

It's turn to use the ToString we saw here.

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TypeSynonymInstances #-}
import qualified Data.Text as T
import qualified Data.Text.IO as T
import qualified Data.ByteString.Char8 as C
import qualified Data.ByteString as B
import qualified Network.HTTP as H
import Data.String (fromString, IsString)

main = do
  putStrLn $ urlEncode ("hello world!" :: String)
  T.putStrLn $ urlEncode ("hello world!" :: T.Text)
  B.putStrLn $ urlEncode ("hello world!" :: B.ByteString)

class ToString a where
  toString :: a -> String

instance ToString String where
  toString = id

instance ToString T.Text where
  toString = T.unpack

instance ToString B.ByteString where
  toString = C.unpack

urlEncode :: IsString a => ToString a => a -> a
urlEncode = fromString . H.urlEncode . toString

This urlEncode function defined here can use String, ByteString and Text just by the single function! I just combined IsString and ToString there :)

Footnote

*1 see also: http://www.slideshare.net/ujihisa/text-manipulation-withwithout-parsec

Completing Method Names in Ruby Based on Input and Output

2011-10-05T22:19:00.001-07:00

I described how to auto-complete method names of an object on Vim but here let me describe about a bit more restricted one. How can we auto-complete method names of an object that goes to a given object?

An example here that you send succ or next on an object 1 when you want to get an object 2.

1.succ #=> 2

Another example. first method will let you get an object :a by [:a, :b, :c].

[:a, :b, :c].first #=> :a

It is natural that you expect Vim automatically completes the method names when the cursor is on the . just after the object.

neco-rubymf.vim

A neocomplcache plugin, neco-rubymf, does exactly what I described above.

Installation:

Install a Rubygems package methodfinder.
```
$ gem install methodfinder
```
Install the neocomplcache plugin neco-rubymf. If your Vim still don't have neocomplcache, install it as well.

Try inputting the following line in a buffer where the 'filetype' is ruby. Note that ¶ means cursor here.

1.¶ #=> 2

That shows methods next and succ in completion popup automatically, which the method an object 1 owns and also the returning value is 2.

Similarly,

[:a, :b, :c].¶ #=> :a

gives this.

The screenshots below are other examples.

※The last example occurs stochastically.

Benchmarks between Clang and GCC about Ruby 1.9.3

2011-08-03T21:58:00.001-07:00

Since Matz Ruby Implimentation ruby 1.9.3 has been released as preview release, I think that all Rubyists should abandon the ruby they are usually using, ruby 1.9.2, and should use 1.9.3 now.

Ruby 1.9.3 also supports clang instead of gcc inofficially. You can compile ruby with clang 3.0 and almost everything works(*1).

Building clang 3.0

There are two things you have to understand before you build clang.

Clang is a subproject of LLVM. You cannot build clang independently to LLVM unless you are an LLVM specialist.
LLVM build directly has to be not in LLVM source tree for some reason.

Otherwise it just fails building.

$ mkdir -p ~/src/llvm-git-build
$ cd ~/git
$ git clone http://llvm.org/git/llvm.git
$ cd tools
$ git clone http://llvm.org/git/clang.git
$ cd ~/src/llvm-git-build
$ ~/git/llvm/configure --prefix=`pwd`/local
$ make && make install

That will install llvm commands and clang in ~/src/llvm-git-build/local/bin directory.

$ ~/src/llvm-git-build/local/bin/clang -v
clang version 3.0 (http://llvm.org/git/clang.git d484040a5fc8d8d8a7a6a0239fc3b34486258181)
Target: x86_64-apple-darwin10.8.0
Thread model: posix

Building ruby 1.9.3 with clang

$ cd ~/git
$ git clone http://github.com/ruby/ruby.git ruby193
$ cd ruby193
$ git checkout ruby_1_9_3 origin/ruby_1_9_3

a.sh

export CC=~/src/llvm-git-build/local/bin/clang
autoconf && ./configure --with-out-ext=tk\* --with-arch=x86_64 --prefix=`pwd`/local --with-readline-dir=/usr/local --with-libyaml-include=/usr/local/include && make && make install

then

$ sh a.sh

Benchmark

$ local/bin/ruby benchmark/run.rb --ruby=local/bin/ruby

I ran on ruby made by clang and ruby made by gcc and compared them.

	gcc/clang	gcc [sec]	clang [sec]
loop_generator	1.55	1.187	0.768
vm_thread_pass_flood	1.38	0.52	0.377
vm1_const	1.23	1.449	1.175
so_random	1.15	1.191	1.04
so_k_nucleotide	1.11	0.02	0.018
vm2_case	1.10	0.338	0.307
vm2_regexp	1.07	1.8	1.683
loop_whileloop	1.07	0.677	0.634
vm_thread_pipe	1.07	1.659	1.556
so_count_words	1.06	0.442	0.418
loop_whileloop2	1.06	0.148	0.14
so_reverse_complement	1.06	0.019	0.018
vm1_swap	1.06	0.956	0.906
vm2_array	1.05	1.543	1.468
vm1_ivar	1.04	1.662	1.596
vm1_length	1.04	1.467	1.412
vm_thread_create_join	1.03	4.45	4.321
vm_thread_pass	1.02	2.308	2.258
loop_times	1.02	1.834	1.796
app_uri	1.01	1.362	1.346
io_select2	1.00	3.976	3.957
vm1_rescue	1.00	0.799	0.798
vm1_ivar_set	1.00	1.766	1.765
io_select3	1.00	0.03	0.03
vm1_ensure	1.00	0.736	0.738
io_select	1.00	3.503	3.513
vm_thread_alive_check1	1.00	0.327	0.328
so_mandelbrot	0.99	6.776	6.817
vm1_not	0.99	0.958	0.965
vm2_poly_method_ov	0.98	0.422	0.43
io_file_write	0.98	1.78	1.815
so_nbody	0.97	4.861	5.012
vm2_eval	0.97	28.817	29.772
so_partial_sums	0.97	6.003	6.211
so_concatenate	0.96	5.322	5.536
so_exception	0.96	1.653	1.723
loop_for	0.95	1.865	1.96
vm1_neq	0.95	1.206	1.269
io_file_read	0.95	3.764	3.965
so_sieve	0.94	1.06	1.13
vm3_gc	0.93	1.593	1.712
so_pidigits	0.92	0.905	0.982
so_spectralnorm	0.92	3.911	4.265
vm2_proc	0.91	0.858	0.941
so_nested_loop	0.91	1.415	1.554
so_fannkuch	0.89	2.321	2.6
vm3_clearmethodcache	0.89	0.683	0.767
so_array	0.89	1.859	2.098
vm2_defined_method	0.88	4.449	5.034
io_file_create	0.88	4.266	4.831
vm1_simplereturn	0.88	2.034	2.307
vm_thread_mutex3	0.88	1.894	2.164
vm2_send	0.87	0.494	0.567
vm2_unif1	0.87	0.419	0.484
so_matrix	0.86	1.173	1.357
so_nsieve_bits	0.86	4.002	4.637
app_tak	0.86	1.133	1.316
so_lists	0.85	1.439	1.698
app_tarai	0.85	0.9	1.062
vm1_block	0.85	2.818	3.331
so_ackermann	0.82	0.891	1.082
vm_thread_mutex1	0.82	1.145	1.393
so_nsieve	0.80	3.188	3.967
vm2_super	0.80	0.68	0.852
so_binary_trees	0.79	0.492	0.62
app_strconcat	0.79	1.93	2.444
vm2_mutex	0.79	1.546	1.968
vm2_zsuper	0.78	0.682	0.88
vm2_poly_method	0.77	2.654	3.444
app_factorial	0.76	1.317	1.734
app_erb	0.76	1.834	2.426
so_meteor_contest	0.75	4.953	6.626
so_object	0.74	1.047	1.416
vm2_method	0.71	2.174	3.049
app_pentomino	0.70	25.525	36.325
app_fib	0.70	0.812	1.168
so_fasta	0.67	2.866	4.302
app_answer	0.66	0.075	0.113
app_mandelbrot	0.65	2.422	3.743
app_raise	0.49	0.861	1.743
vm_thread_mutex2	0.41	1.228	2.977

Considering all the results gcc-made ruby is faster than clang-made ruby but not in all the benchmarks. Hopefully the results must be an interesting example for some people.

Footnote and references

*1: make test-all shows that there is one issue in Fiber with the ruby made by clang.
http://ujihisa.blogspot.com/2011/01/how-to-build-mri-n-times-faster.html comparison between clang and gcc about time to build ruby

Compiling a C File, Preserving the LLVM File

2011-07-25T21:14:00.001-07:00

Assuming you have a C file a.c and want to get a.ll and a.o.

An intuitive way is that

$ clang -flto -S a.c
$ mv a.s a.ll
$ clang -flto -c a.c

But the second clang looks waste of CPU; you generate an LLVM assembly language and then you generate an object file through LLVM assembly language without using the generated assembly file.

Let me break down the second command into small pieces..

$ clang -flto -S a.c
$ mv a.s a.ll
$ llc a.ll
$ gcc -c a.s

In GCC

$ gcc -c --save-temps a.c

oh... very easy...

http://www.qiita.com/questions/87

String Interpolation in Haskell

2011-07-23T14:18:00.001-07:00

Ruby has a special syntac in String literal which name is String Interpolation that you can write the result of a expression in a string easily.

"hello, #{name}!"

The notation is equivalent to the following notation.

"hello, " + name + "!"

That's not only available in Ruby but also in CoffeeScript.

I wrote this for Haskell in Haskell.

{-# LANGUAGE OverloadedStrings #-}
import qualified Data.Text as T
import qualified Data.Text.IO as T
import qualified Data.Attoparsec.Text as P
import Control.Applicative

main = T.putStrLn $ stringInterpolate "as{df \"jkl#{x}jkw\"jl}kf"

(<<) = T.append

stringInterpolate :: T.Text -> T.Text
stringInterpolate x = case P.parseOnly stringInterpolate' x of
                           Left x -> T.pack x
                           Right x -> x

stringInterpolate' :: P.Parser T.Text
stringInterpolate' = P.try $
  T.concat <$> P.many (block <|> string <|> T.singleton `fmap` P.notChar '}')

block :: P.Parser T.Text
block = P.try $ do
  x <- (P.char '{') *> stringInterpolate' <* (P.char '}')
  return $ "{" << x << "}"

-- "jklsfj#{...}sdjfkl"
string :: P.Parser T.Text
string = P.try $ do
  x <- (P.char '"') *> stringContent <* (P.char '"')
  return $ "(\"" << x << "\")"

-- jklsfj#{...}sdjfkl
stringContent :: P.Parser T.Text
stringContent = P.try $
  T.concat <$> P.many (interpolate <|> T.singleton `fmap` P.notChar '"')

-- #{...}
interpolate :: P.Parser T.Text
interpolate = P.try $ do
  x <- P.string "#{" *> stringInterpolate' <* P.char '}'
  return $ "\" ++ " << x << " ++ \""

String concatination in Haskell is ++. The program above converts string literals like the below.

"hello, #{name}!"

("hello, " ++ name ++ "!")

You can write arbitrary expression in the placeholder.

"hello, #{[ toUpper c | c <- ['a'..'z'] ]}!"

("hello, " ++ [ toUpper c | c <- ['a'..'z'] ] ++ "!")

Vanrb Lightning Talk Slides: Ruby and Vim

2011-06-30T19:39:00.001-07:00

http://www.meetup.com/vancouver-ruby-rails/events/21277581/

structure

RSense

A Ruby development toolset

Code completion
Type inspection
Definition jump

RSense code completion

a.rb

a = '123'
b = a.to_i
b.

rsense command

$ rsense code-completion --file=a.rb --location=3:2
completion: to_enum Object#to_enum Object METHOD
completion: succ! String#succ! String METHOD
completion: swapcase! String#swapcase! String METHOD
completion: instance_variable_defined? Object#instance_variable_defined? Object METHOD
...

Implementation

Front-end command bin/rsense
Back-end server process with Java&JRuby
Type-inference algorithm "modified-Cartesian Product Algorithm"

limitations

classes in the buffer
1.8 built-in libraries
incorrect syntax
evals and exceptions

Vim

Vim and me

fullscreen MacVim
vimshell (no other terminals)
neocomplcache
unite
quickrun

RSense built-in Vim plugin

User-defined completion <C-x><C-u>

a = '123'
b = a.to_i
b._

Insert-mode completions in general

<C-n> Keyword
<C-x><C-f> File name
<C-x><C-s> English spelling
<C-x><C-o> Omni
<C-x><C-u> User-defined

neocomplcache

Insert-mode auto-completion plugin framework

No key mapping necessary
Integrates lots of completions

RSense as omni func

~/.vimrc

let g:rsenseUseOmniFunc = 1
let g:rsenseHome = expand('~/git/rsense')

let g:neocomplcache_omni_patterns = {}
let g:neocomplcache_omni_patterns.ruby = '[^. *\t]\.\w*\|\h\w*::'

one more thing

neco-: prefix for neocomplcache plugins
neco-rake: rake task completion

structure

http://ujihisa.blogspot.com
http://twitter.com/ujm
google rsense
google neocomplcache if you are a Vimmer

String#slice of Ruby in Vim script and Haskell

2011-06-12T00:57:00.001-07:00

A Ruby example of String#[]

str = "Oh, this is a pen."
p str[/this is a (\w+)\./, 1]

The result is "pen". Since String#[] is just an alias of String#slice, (*1)

p str.slice(/this is a (\w+)\./, 1)

The result is completely same.

in Vim script

Vim script version needs two separate processes; getting the list of first-matched string itself and containing sub matches, and then getting the specific item.

let str = "Oh, this is a pen."
echo matchlist(str, 'this is a \(\w\+\)\.')[1]

(Added at Sun Jun 12 09:26:44 PDT 2011) thinca suggested that matchstr() with \zs and \ze is very handy, particularly because of the different behavior in the case when it didn't match.

let str = "Oh, this is a pen."
echo matchstr(str, 'this is a \zs\w\+\ze\.')

in Haskell

Haskell version needs three separate processes with Text.Regex.Posix.=~; it's almost same to Vim but the default =~ behaviour is to assume the regex object has "global" option, so you have to pick which match.

import Text.Regex.Posix ((=~))
main = do
  let str = "Oh this is a pen."
  print $ head (str =~ "this is a ([a-zA-Z_]*)" :: [[String]]) !! 1

(Added at Sun Jun 12 12:54:01 PDT 2011) The following code is another example; it's safe in runtime and also this supports Vim's \zs and \ze.

import Text.Regex.PCRE ((=~))
import Data.String.Utils (replace)
import Safe (headMay, atMay)
import Data.Maybe (fromMaybe)

matchstr :: String -> String -> String
matchstr expr pat =
  let x = replace "\\zs" "(" $ replace "\\ze" ")" pat in
  fromMaybe "" $ headMay (expr =~ x :: [[String]]) >>= \y -> atMay y 1

main = print $ matchstr "this is a pen" " \\zs\\w+\\ze"

(*1) Precisely no. Try and check the differences between them without the second argument.

A Vim Workshop in Tokyo, Japan in May

2011-05-31T21:17:00.001-07:00

I organized a Vim workshop in Tokyo, Japan. The name of the workshop was ujihisa.vim. More than 40 people tried to come but 20+ people attended at this workshop due to the capacity of the room.

The venue

I went to Tokyo for this workshop.

a photo by guyon

Thanks TMI and kana1!

Talks

shadow.vim with me
- shadow.vim https://github.com/ujihisa/shadow.vim
- arbitrary macros for arbitrary programming languages
- e.g. compiling CoffeeScript into JavaScript secretly
- it's not a framework but just a tool; you can combine it with others easily
- shadow.el is also available https://github.com/mooz/shadow.el
all your vimrc is belong to us with kana1
- picks a victim, points a lot of bad smells in the vimrc of the victim, and fix them!
- if the length of your vimrc is less than 1000, you aren't a Vimmer yet.
- syntax on is evil. use syntax enable.
i18n with niw
- strchars() doesn't treat multi-byte character correctly.
tiny-vim with guyon
- about Debian package "tiny-vim"
- no built-in documentation!
Vim in 5 minutes with Shougo
- neocomplcache, unite, vimshell and vimfiler author
- explained everything about Vim exactly within 5 minutes and extra minutes for further details
vital.vim with me
- it's amazing. use it if you are a Vim plugin author.
- something like jQuery (from JavaScript) + Bundler (from Ruby)
- there's an issue in vital.vim about debugging with quickrun. (*)
metarw-yakiudon with sora_h
- a metarw plugin yakiudon
- works almost exactly same as blogger.vim but the backend Ruby library is different
- after this presentation he talked about Ruby core development

(*) thinca, the quickrun author, fixed this issue with the following change in vital during the workshop!

Misc

Before and after the workshop some of the attendees had lunch and dinner with good Vim conversation. The topics were a lot.. vimshell, CoffeeScript, Titanium, HootSuite, Twitter, Ruby core, the ultimate programming language Zimbu, PHP vs Vim script, Emacs lisp and etc...

Quotes

"It's... it's not the Vim I knew..."
"I came to Japan from San Francisco only for attending at this workshop"
"Wow! HootSuite for iPad works amazingly!!!"

Workshops in Amagasaki, Japan by Me

2011-05-09T23:16:00.001-07:00

I organized the 7th Vim workshop and a CoffeeScript workshop in Amagasaki, Japan on the last Saturday.

About 20 to 30 people attended the workshops. Even though the room had an air conditioner, the room was so hot and humid like sauna by them (and their MacBooks.)

I was supposed to make presentations with demonstrations, but unfortunately I forgot bringing my mini-DVI port for my initial version of MacBook Air. There were many MacBook Air attendees but nobody had the same version to me. I eventually gave up using my computer and just made presentation by friends' computers. Sorry for missing demonstrations.

Vim Workshop#7

There were five talks

Recommended Vim plugins (by me)
The implementation of lingr.vim (by tsukkee)
Bundle with Vundle (by kozo2)
Unite is useful, and rails.vim as well (by Sixeight)
All about vital.vim (by me)

I broadcasted them and recorded some of them (I just forgot recording the others.) http://ustream.tv/channel/uji

CoffeeScript Workshop

There were five talks.

Introduction to CoffeeScript -- syntax and usage (by me)
Node.js on Windows, and introduction to no.de (by cuzic)
iPhone app development with CoffeeScript and Titanium (by deguchi)
TDD with Jasmine and CoffeeScript (by http://twitter.com/nanki)
Details of CoffeeScript syntax (by yhara and me)

by fukui

Vim 7.3 Conceal Current Issue

2011-02-22T08:12:00.001-08:00

Vim 7.3 new feature Conceal has an issue that when you change your colorscheme the current conceal information will be partly removed automatically.

I found this issue with the combination of Haskell-Conceal plugin and unite-colorscheme plugin which internally uses tabpagecolorscheme plugin. Every time I moved the tab, the concealed text highlighted wrongly. After a research I figured out that it was not by changing the tab but by changing colorscheme into the same colorscheme.

First of all, the idiom to define a concealed token in a syntax file is (1) to define a syntax match {your favourite name here} "{regexp}" conceal cchar={the replacement} (2) and to link the color by highlight! link Conceal {token name}. Note that normal highlights only needs to hightlight without the bang.

The main part of the implementation of colorscheme command is defined as below which is from src/syntax.c of vim repository.

6755 /*
6756  * Load color file "name".
6757  * Return OK for success, FAIL for failure.
6758  */
6759     int
6760 load_colors(name)
6761     char_u *name;
6762 {
6763     char_u *buf;
6764     int        retval = FAIL;
6765     static int recursive = FALSE;
6766 
6767     /* When being called recursively, this is probably because setting
6768      * 'background' caused the highlighting to be reloaded.  This means it is
6769      * working, thus we should return OK. */
6770     if (recursive)
6771    return OK;
6772 
6773     recursive = TRUE;
6774     buf = alloc((unsigned)(STRLEN(name) + 12));
6775     if (buf != NULL)
6776     {
6777    sprintf((char *)buf, "colors/%s.vim", name);
6778    retval = source_runtime(buf, FALSE);
6779    vim_free(buf);
6780 #ifdef FEAT_AUTOCMD
6781    apply_autocmds(EVENT_COLORSCHEME, NULL, NULL, FALSE, curbuf);
6782 #endif
6783     }
6784     recursive = FALSE;
6785 
6786     return retval;
6787 }

Basically this function runs :source colors/{the colorscheme name}.vim and them runs autocmds already registers in Colorscheme event. This function itself looks good.

Every colorscheme files run :highlight clear in the beginning of the files. This removes all highlights except for user-defined ones. Thus it preserves syntactic information of each syntax files, but breaks conceal information given by each syntax files, because it's not treated as user-defined one but as just extended built-in one.

Possible solutions

I propose the following solutions. Each of them are independent and exclusive.

Remove the procedure of removing conceal information from :highlight clear command
Fix colorscheme command to recover the conceal information after changing colorscheme
The behaviour is specification. Just add documentation in Vim core to persuade conceal users to add autocmd event to change conceal again after colorscheme changed.

The below is the patch for the solution (a).

diff --git src/syntax.c src/syntax.c
index 369311f..90165cf 100644
--- src/syntax.c
+++ src/syntax.c
@@ -6572,10 +6572,6 @@ static char *(highlight_init_light[]) =
  CENT("ColorColumn term=reverse ctermbg=LightRed",
       "ColorColumn term=reverse ctermbg=LightRed guibg=LightRed"),
 #endif
-#ifdef FEAT_CONCEAL
-   CENT("Conceal ctermbg=DarkGrey ctermfg=LightGrey",
-        "Conceal ctermbg=DarkGrey ctermfg=LightGrey guibg=DarkGrey guifg=LightGrey"),
-#endif
 #ifdef FEAT_AUTOCMD
  CENT("MatchParen term=reverse ctermbg=Cyan",
       "MatchParen term=reverse ctermbg=Cyan guibg=Cyan"),
@@ -6662,10 +6658,6 @@ static char *(highlight_init_dark[]) =
  CENT("MatchParen term=reverse ctermbg=DarkCyan",
       "MatchParen term=reverse ctermbg=DarkCyan guibg=DarkCyan"),
 #endif
-#ifdef FEAT_CONCEAL
-   CENT("Conceal ctermbg=DarkGrey ctermfg=LightGrey",
-        "Conceal ctermbg=DarkGrey ctermfg=LightGrey guibg=DarkGrey guifg=LightGrey"),
-#endif
 #ifdef FEAT_GUI
  "Normal gui=NONE",
 #endif

Delegation in Vim script

2011-02-05T20:04:00.001-08:00

In case when you define a function F1 which behaves same as a given function F2 in Vim script, you may just choose the following cases depending on the arguments of F2.

The solution is just to use call().

function! F1(...)
  return call('F2', a:000)
endfunction

But if you also want to preserve the definition of arguments? There are still solution of each cases.

Named arguments (or no arguments)

function! F1(x)
  return F2(a:x)
endfunction

Unnamed arguments

function! F1(...)
  return call('F2', a:000)
endfunction

Then you can make mixed version, using call().

function! F1(x, ...)
  return call('F2', insert(deepcopy(a:000), a:x))
endfunction

Basically this is "cons"ing a named variable into the top of the list.

Summary

call() is so handy.

Shougo Matsu, the neocomplcache and unite author, suggested me to use call() instead of the following "First version" of this blog post. That's absolutely better idea than I had. Thanks Shougo!

First version

The following sentense was legacy. I leave them just for your additional information.

Named arguments (or no arguments)

function! F1(x)
  return F2(a:x)
endfunction

Unnamed arguments

function! F1(...)
  return eval('F2(' . join(map(range(1, a:0), '"a:" . v:val'), ', ') . ')')
endfunction

This looks more difficult than it should be. Let me explain it.

a:0 is the number of the arguments of the function. If you call F1(x, y, z), a:0 is 3. You can access each arguments as a:1, a:2 and a:3.

range(1, a:0)

This becomes a list [1, 2, 3] if the number of the arguments of F1 is 3.

map(range(1, a:0), '"a:" . v:val'), ', ')

This becomes 'a:1, a:2, a:3'. Then you can make a string "F2(a:1, a:2, a:3)" dynamically, and call it by eval.

Then you can make mixed version easily.

function! F1(x, ...)
  return eval('F2(a:x, ' . join(map(range(1, a:0), '"a:" . v:val'), ', ') . ')')
endfunction

The difference is only just "F2(a:x, " which was originally "F2(".

How To Build MRI n Times Faster

2011-01-27T20:19:00.001-08:00

Clang, a compiler front end which you can use instead of gcc, can build MRI: Matz Ruby Implementation, only if you succeeded in avoiding all pitfalls.

Build the latest LLVM and Clang

Mac OS X Snow Leopard has builtin clang command but it's too old to build ruby. First of all, get the latest clang and it's dependency, the latest LLVM.

$ svn checkout http://llvm.org/svn/llvm-project/llvm/trunk llvm && cd llvm/tools && svn checkout http://llvm.org/svn/llvm-project/cfe/trunk clang && cd ..
$ ./configure && time make

Note that it takes really long time. I recommend you not to build before you go away from keyboard for a while but to build before you go to bed.

A simple test

hello.c

#include<stdio.h>
int main() {
  puts("Hello, world!");
  return 0;
}

building on gcc and clang

$ time gcc hello.c -o hello-gcc
!!!        0.34 real         0.01 user         0.03 sys!!!

[%] /private/tmp
$ time ~/src/llvm/Debug+Asserts/bin/clang hello.c -o hello-clang
!!!        0.11 real         0.06 user         0.02 sys!!!

clang is 3 times faster than gcc to build the hello world code. The executable files work samel.

Ruby

On GCC

$ git clone https://github.com/ruby/ruby.git ruby193-gcc
$ cd ruby193-gcc
$ autoconf
$ ./configure --prefix=`pwd`/local --enable-pthread --disable-install-doc --with-out-ext=tk\* --program-suffix=193 --with-readline-dir=/usr/local --with-arch=x86_64
$ time make
...
!!!      214.93 real       191.58 user        23.57 sys!!!

On Clang

$ git clone https://github.com/ruby/ruby.git ruby193
$ cd ruby193
$ ./configure --prefix=`pwd`/local --enable-pthread --disable-install-doc --with-out-ext=tk\* --program-suffix=193 --with-readline-dir=/usr/local PATH=/Users/ujihisa/src/llvm/Debug+Asserts/bin/:$PATH CC=/Users/ujihisa/src/llvm/Debug+Asserts/bin/clang CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments --with-arch=x86_64
$ time make
!!!      932.06 real       883.44 user        31.09 sys!!!

It took 3 min 34.93 sec by GCC while it took 15 min 32.06 sec by Clang. GCC is 4.34 times faster than Clang to build MRI.

Pitfalls

If you use clang command of Mac OS X Snow Leopard, even though you'll succeed in finishing make ruby 1.9.3, but you cannot do anything by the Ruby.

$ ./local/bin/ruby -ve 1
ruby 1.9.3dev (2011-01-25 trunk 30651) [x86_64-darwin10.6.0]
!!!dyld: lazy symbol binding failed: Symbol not found: _rb_encdb_declare!!!
!!!  Referenced from: /Users/ujihisa/git/ruby193/local/lib/ruby/1.9.1/x86_64-darwin10.6.0/enc/encdb.bundle!!!
!!!  Expected in: flat namespace!!!

!!!dyld: Symbol not found: _rb_encdb_declare!!!
!!!  Referenced from: /Users/ujihisa/git/ruby193/local/lib/ruby/1.9.1/x86_64-darwin10.6.0/enc/encdb.bundle!!!
!!!  Expected in: flat namespace!!!

!!!/Users/ujihisa/git/ruby193/local/lib/ruby/1.9.1/x86_64-darwin10.6.0/enc/encdb.bundle: [BUG] Segmentation fault!!!
!!!ruby 1.9.3dev (2011-01-25 trunk 30651) [x86_64-darwin10.6.0]!!!

!!!-- Control frame information -----------------------------------------------!!!
!!!c:0002 p:-537663266 s:0004 b:0004 l:000003 d:000003 TOP   !!!
!!!c:0001 p:0000 s:0002 b:0002 l:001dd8 d:001dd8 TOP   !!!


!!!-- See Crash Report log file under ~/Library/Logs/CrashReporter or ---------!!!
!!!-- /Library/Logs/CrashReporter, for the more detail of ---------------------!!!
!!!-- C level backtrace information -------------------------------------------!!!

!!!-- Other runtime information -----------------------------------------------!!!

!!!* Loaded script: ./local/bin/ruby!!!

!!!* Loaded features:!!!

!!!    0 enumerator.so!!!

!!![NOTE]!!!
!!!You may have encountered a bug in the Ruby interpreter or extension libraries.!!!
!!!Bug reports are welcome.!!!
!!!For details: http://www.ruby-lang.org/bugreport.html!!!


!!!vimshell: signal 6(SIGABRT) "./local/bin/ruby -ve 1"!!!

Conclusion

You can build Ruby 1.9.3 by Clang. It's 0.23 times faster than building by GCC.

A Contributed Article to Kernel/VM Advent Calendar: VIM=VM

2011-01-05T09:27:00.001-08:00

There are two kinds of programmers; one uses Vim and the other uses Emacs. I don't think there are positive counter opinions about that Emacs is not an editor but an environment. On the other hand, there are some opinions that Vim is an editor or not, even though everyone agrees with the fact that vi is an editor. Some people, including the author of this article, claim that Vim is a virtual machine. Here I give you instructions of a Vim plugin shadow.vim instead of explaining why Vim is VM directly.

shadow.vim

The Vim plugin shadow.vim supports wrapping a virtual file with a program automatically with thin configuration.

Here it's a quote from a programmer:

"Nobody knows the Java code you committed is originally written in Scheme."

Usage

Assuming the product is a.pl, create a.pl.shd first.

a.pl (in Vim):

## ruby -e 'puts $<.read.gsub(/$/, ";")'
$a = 1
print($a)

Open a.pl in Vim. The Vim actually shows the contents of a.pl.shd. When you save the file, the command in the first line without ## runs, then the actual a.pl will be the result.

a.pl (actually):

$a = 1;
print($a);

Install

Unarchive the zip file into a directory that is under &rtp of your Vim, which stands for run time path, including ~/.vim dir.

Use Case

Here there are three examples, but you can use more general purposes.

Commit JavaScript files which was written in CoffeeScript

before

    ## coffee -csb
    f = (x) -> x + 1
    print f 10

    # vim: set ft=coffee :

after

    var f;
    f = function(x) {
      return x + 1;
    };
    print(f(10));

Use cpp command before committing Java files.
Markdown, Haml or something else to HTML

More examples

You can write code in C and save the file as GNU Assembly Language on your Linux automatically, so you can avoid portability.

References

VIM=VM http://www.slideshare.net/Shougo/vim-presentation-5330807 by Shougo
http://atnd.org/events/10701

Búsqueda/Instalación Librerías RubyGems Fácilmente

2010-12-13T17:49:00.001-08:00

¿Cómo se puede explorar y tratar de librerías RubyGems? Hay una gran cantidad de joyas como las estrellas en el cielo. Si lo desea, estar de pie sobre los hombros de gigantes.

Una escena típica de cuando usted está buscando una librería que cumple su trabajo sería como el siguiente, por ejemplo, que busca una librería que está relacionado con DataMapper.

$ gem search dm

No consigues nada porque se le olvidó especificar -r opción.

$ gem search dm -r

Usted obtiene demasiados resultados, porque hay demasiadas joyas que incluyen "dm" en el nombre, por ejemplo "admin" algo.

De todos modos, que finalmente encontró una buena librería cuyo nombre es "dm-is-persistent_state_machine".

 $ gem install dm-is-persistent_states_machine

Este failes porque el nombre de la librería correcta es "dm-is-persistent_state_machine", mientras que ha escrito "dm-is-persistent_states_machine". Sí, eso es muy difícil de escribir correctamente cuando el nombre de la librería es muy larga. Puede ser difícil si el nombre no está en su lengua materna. (*1)

unite-gem

Una solución es usar un plugin Unite unite-gem. Después de instalar este plugin, basta con ejecutar

 :Unite gem

y el tipo nada. Verá los siguientes resultados.

y se puede reducir el número de candidatos.

a continuación, simplemente pulse <Cr>... el proceso de instalación se produjo. Usted no tiene que introducir el nombre completo de la librería.

Nota

(*1): por ejemplo, muchas personas trataron de tipo "nokogiri", pero mal escrita como "nokogirl". Por último los autores Nokogiri realizó una joya alias "nokogirl" y todo el mundo se puso feliz. Este blog es sólo givin una versión más general de la solución.

How To Write And Show Multibyte Characters With Snap Framework

2010-12-13T09:10:00.001-08:00

I'm playing with Snap these days. Snap is a web framework for Haskell.

The default template of application code will be like below

{-# LANGUAGE OverloadedStrings #-}
module Main where

import           Control.Applicative
import           Snap.Types
import           Snap.Util.FileServe
import           Text.Templating.Heist
import           Text.Templating.Heist.TemplateDirectory

import           Glue
import           Server


main :: IO ()
main = do
    td <- newTemplateDirectory' "templates" emptyTemplateState
    quickServer $ templateHandler td defaultReloadHandler $ \ts ->
        ifTop (writeBS "hello") <|>
        route [ ("foo", writeBS "bar")
              , ("echo/:echoparam", echoHandler)
              ] <|>
        templateServe ts <|>
        dir "static" (fileServe ".")


echoHandler :: Snap ()
echoHandler = do
    param <- getParam "echoparam"
    maybe (writeBS "must specify echo/param in URL")
          writeBS param

See the following line in main function:

        ifTop (writeBS "hello") <|>

This just shows a text written in ByteString "hello" to browser.

$ cabal install
$ {your application name} 3000
$ curl http://localhost:3000
hello

You can change the message "hello" to something and show the message on browser, but if the text is non-ascii like utf-8, you have to change other places as well. See the following patch:

diff --git snapfib.cabal snapfib.cabal
index d7a98fe..35b5c92 100644
--- snapfib.cabal
+++ snapfib.cabal
@@ -28,7 +28,8 @@ Executable snapfib
     text,
     containers,
     MonadCatchIO-transformers,
-    filepath >= 1.1 && <1.2
+    filepath >= 1.1 && <1.2,
+    utf8-string

   if impl(ghc >= 6.12.0)
     ghc-options: -threaded -Wall -fwarn-tabs -funbox-strict-fields -O2
diff --git src/Main.hs src/Main.hs
index d5b24c4..036b2db 100644
--- src/Main.hs
+++ src/Main.hs
@@ -9,13 +9,14 @@ import           Text.Templating.Heist.TemplateDirectory

 import           Glue
 import           Server
+import           Data.Text


 main :: IO ()
 main = do
     td <- newTemplateDirectory' "templates" emptyTemplateState
     quickServer $ templateHandler td defaultReloadHandler $ \ts ->
-        ifTop (writeBS "hello") <|>
+        ifTop (writeText $ pack "こんにちはこんにちは!") <|>
         route [ ("foo", writeBS "bar")
               , ("echo/:echoparam", echoHandler)
               ] <|>

The function writeBS in Snap receives ByteString and shows the message to browser as a response. This cannot handle UTF-8 strings for some reason, so instead of using writeBS with ByteString you have to use writeText with Text represented message. You can convert into Text with Data.Text.pack function.

Ruby-like Object-oriented Notation in Haskell

2010-12-08T01:19:00.001-08:00

This article doesn't discuss about something practical but about something experimental.

import Prelude hiding ((.))

send :: Object -&gt; Method -&gt; Object
send (Fixnum value) (Method "to_s") = Str $ show value
send _ _ = undefined

(.) = send

data Object = Fixnum Int | Str String deriving Show
data Method = Method String deriving Show

main = print $ x.to_s
  where
    x = Fixnum 10
    to_s = Method "to_s"

See the line in main function definition: print $ x.to_s. This looks like Ruby.

You can hide the definition of Haskell Prelude "." and can define your own function which name is "."; here I defined send first then made an alias of it.
Object and sending method into an object are like just a data and a function which receives an argument of a specific type unless you think about class or prototype; or just, say, "performance."

Writing Fibonacci Web Server in Haskell

2010-12-06T19:02:00.001-08:00

A practice: implementing a web service which receives a number by URL parameter and returns the corresponding Fibonacci sequence number.

GET
http://localhost:8000/fib/10

returns

fib(10) = 55

I used Snap Framework.

$ mkdir snapfib
$ cd snapfib
$ snap init

This generates an empty snap project files. then

$ cabal install
$ snapfib

builds the codes and installs a command snapfib in your ~/.cabal/bin/snapfib. You may run the app locally with the default port 8000; you may open the default page, hello world, on your browser.

Then implement fib sequence, a utility function, and the controller/view. This allows you to get the Fibonacci sequence number just by URL parameter.

Below is the main routine code from the repository.

{-# LANGUAGE OverloadedStrings #-}
module Main where

import           Control.Applicative
import           Snap.Types
import           Snap.Util.FileServe
import           Text.Templating.Heist
import           Text.Templating.Heist.TemplateDirectory

import           Glue
import           Server
import           Data.ByteString.Char8 (pack, unpack)

fibs :: [Integer]
fibs = 0 : 1 : zipWith (+) fibs (tail fibs)

fibInString :: Int -> String
fibInString n = "fib(" ++ show n ++ ") = " ++ show (fibs !! n)

main :: IO ()
main = do
    td <- newTemplateDirectory' "templates" emptyTemplateState
    quickServer $ templateHandler td defaultReloadHandler $ \ts ->
        ifTop (writeBS "hello world") <|>
        route [ ("foo", writeBS "bar")
              , ("fib/:n", fibHandler)
              ] <|>
        templateServe ts <|>
        dir "static" (fileServe ".")


fibHandler :: Snap ()
fibHandler = do
    param <- getParam "n"
    maybe (writeBS "must specify fib/n in URL")
          (writeBS . pack . fibInString . read . unpack) param

(writeBS . pack . fibInString . read . unpack) param is converting param :: Maybe ByteString into Int, getting a Fibonacci sequence number in String, converting into ByteString again, and passing it to writeBS function which is defined in Snap Framework.

It was not difficult or complicated to implement such a simple web service in Haskell as long as you have basic Haskell knowledges like Maybe Monad or String manipulations. The problem was, in my opinion, it took long time to build the web app. Every time you fix your code, you have to wait for the compilation before you access your web service.

Investing The Methods Of An Object On Unite

2010-12-02T17:23:00.001-08:00

How often do you use Object#methods? That's convenient particularly on IRB.

methods on irb

This is also available on Unite.vim. Write the following code on your ~/.vimrc

let s:unite_source = {
      \ 'name': 'evalruby',
      \ 'is_volatile': 1,
      \ 'required_pattern_length': 1,
      \ 'max_candidates': 30,
      \ }

function! s:unite_source.gather_candidates(args, context)
  if a:context.input[-1:] == '.'
    let methods = split(
          \ unite#util#system(printf('ruby -e "puts %s.methods"', a:context.input[:-2])),
          \ "\n")
    call map(methods, printf("'%s' . v:val", a:context.input))
  else
    let methods = [a:context.input]
  endif
  return map(methods, '{
        \ "word": v:val,
        \ "source": "evalruby",
        \ "kind": "command",
        \ "action__command": printf("!ruby -e \"p %s\"", v:val),
        \ }')
endfunction

call unite#define_source(s:unite_source)

Execute :Unite evalruby on your Vim, and write an expression.

unite

Ruby Advent Calendar jp-en: 2010

This entry is for the event Ruby Advent Calendar jp-en: 2010. The previous post was from yhara and the next post will be from authorNari. I'm sure he will write an entry about GC.

I also had written an entry of the event last year http://ujihisa.blogspot.com/2009/11/write-implementation-and-spec-on-same.html. Time flees away without delay.

Ruby 1.9.2 in Production with Tatsuhiro Ujihisa

2010-11-23T20:14:00.001-08:00

(This is the presentation slides set for vancouver ruby meetup)

Before the talk...

two questions
easy one and normal one

Question 1

An easy Ruby quiz: How to define a method without using "def" keyword? (only with pure Ruby)

def a
  :something
end
p a #=> :something

Sample answer

self.class.module_eval do
  define_method(:a) do
    :something
  end
end
p a

Or

eval 'def a; :something; end'
p a

There's no keyword def but string containing 'def'

I think there are a lot more answers

Question 2

A normal Ruby quiz: How to assign a value to a local variable without using assignment operator "="? (only with pure Ruby)

a = :something
p a #=> :something

Sample answer 1

for a in [:something]; end

p a

"Local variable" is very special in Ruby
Declaring a new local variable is completely static

One more answer which is only on 1.9

Sample answer 2

/(?<a>).*/ =~ ''
eval 'a=:something'

p a

There are a match operator =~ and a string which contains =, but no assignment operator =.

Ruby 1.9.2 in Production with Tatsuhiro Ujihisa

image

This talk contains

What is Ruby 1.9.2?
Differences between 1.8..1.9
Differences between 1.9.1..1.9.2
Differences between 1.9.2..1.9.3

The summary of this talk

"Ruby 1.9.2 makes your code cleaner and easier to maintain."

Ruby versions

Ruby 1.8.6
- Mar 2007
Ruby 1.8.7
- May 2008
Ruby 1.9.0
- Dec 2007
Ruby 1.9.1
- Jan 2009
Ruby 1.9.2
- Aug 2010

Ruby versions

Ruby 1.8.6
- Mar 2007
Ruby 1.9.0
- Dec 2007
- Rails 2.0
Ruby 1.8.7
- May 2008
Ruby 1.9.1
- Jan 2009
Ruby 1.9.2
- Aug 2010
- Rails 3.0

Background: Ruby versions

All of them are officialy stable version
But Ruby 1.9 series didn't look like stable

What's new?

Which problems did 1.9 solved?
How can you write code easily?
-> Examples

Problems and solutions

Verbose hash notation
Difficulty in process handling
etc..

Problem 1

About syntax

Problem 1

JSON is cool. Dictionary in Erlang is cool. Hash in Ruby is...?

JavaScript or Erlang:

{a: 1, b: 2, c: 3}

Ruby:

{:a => 1, :b => 2, :c => 3}

1.9.2 new Hash syntax

This solved the problem

Ruby:

{a: 1, b: 2, c: 3}

This issue mattered particularly in..

Haml, the tool everyone is using, needs a lot of hash.

HTML:

<img src="a.jpg" width="64" height="128"></img>

Haml with ruby variables:

%img{:src => icon_path, :width => icon[:width], :height => icon[:height]}

(the below doesn't work)

%img(src=icon_path width=icon[:width] height=icon[:height])

This issue mattered particularly in..

Using 1.9.2 new hash syntax

%img{src: icon_path, width: icon[:width], height: icon[:height]}

Problem 2

About builtin methods

Problem 2

system or `` operator lacked some important functionalities.

Ruby is a good shell script (Rakefile!),
Ruby has some file/process handling methods,
But..
- You couldn't retrieve the output or error of system
- You could run a command asynchronously with system with "&", but couldn't kill the process directly
- You couldn't run a command asynchronously with ``

Example

Start a Sinatra app by a Ruby script and kill the app by the script

in shell script:

#!/bin/sh
ruby sinatra_app.rb &
PID=$!
# something...
kill $PID

in ruby?

Bad solution

system 'ruby sinatra_app &'

You cannot get the proccess ID, so you cannot kill the process

Better solution

pid = fork do
  exec 'ruby', 'sinatra_app'
end
# something..
Process.kill 'KILL', pid

This doesn't work on NetBSD4 or Windows due to fork()

Best solution

pid = spawn 'ruby', 'sinatra_app'
# something...
Process.kill 'KILL', pid

:)
spawn =~ {fork + exec} or {system + &}
portable

New system and spawn spec

system({'A' => 'b'}, 'c', in: input_io, [:out, :err] => :out)
#=> true/false

spawn(...)
#=> Fixnum

Even more..

open3 standard library

Open3.capture2e
powered by spawn (read the source!)

Problem 3

Local variable shadowing (potantial bug!)

a = :hello
[1, 2, 3].each do |a|
  p a
end
p a #=> 3 (in 1.8)

Problem 4

"Most libraries didn't work"
Yes it was (particularly on 1.9.1)

now?

Active libraries work for sure!

nokogiri
rails

Problem 5

Installation

install ruby, and then install rubygems, ...

rubygems is builtin!

rake as well

Problem 6

Ruby is slow

YARV!

fib 31

1.8.7
- 7.99sec
1.9.2
- 0.64sec
(jruby)
- 3.00sec

How to make legacy code 1.9 compatible?

Changes between 1.8 and 1.9

String isn't Enumerable
when clause doesn't accept colon as semicolon

change between 1.9.1 and 1.9.2

$: doesn't have the current dir.
- require_relative is handy (but long...)

Demo: make a gem library 1.9.2 compatible

github gem library
github create etc..
it's very 1.8 even though it's new
- 0.1.0 was March 3, 2008
- 0.4.5 was Oct 25, 2010; after 1.9.2 public release!

The summary of this talk (again)

"Ruby 1.9.2 makes your code cleaner and easier to maintain."

end

thanks!

Tatsuhiro Ujihisa
@ujm
HootSuite Media, inc
Ruby, Haskell, JavaScript, Vim script, etc..

hootsuite

appendix

JRuby
- --1.9 option
- almost compatible with 1.9 except spawn() or etc

Conditional Operator Associativities

2010-11-23T12:59:00.001-08:00

In Ruby:

a = true ? :a : true ? :b : :c
p a

Guess the answer! Yes, as you thought, the answer is :a.

In JavaScript:

var a = true ? 'a' : true ? 'b' : 'c';
alert(a);

Guess the answer! Yes, as you thought, the answer is a.

In Vim script:

let a = 1 ? 'a' : 1 ? 'b' : 'c'
echo a

Guess the answer! Yes, as you thought, the answer is a.

In PHP:

$a = true ? 'a' : true ? 'b' : 'c';
print_r($a);

Guess the answer! No, the answer if "b".

$a = true ? 'a' : (true ? 'b' : 'c');
print_r($a);

The code above is the equivalent code to the examples in Ruby, JS and Vim script. It's impossibly difficult to imagine why the specification is so.

Something like closure for Vim script

2010-11-18T18:15:00.001-08:00

Vim script doesn't have lambda as a language feature, but you can fake similar thing with the following tricks.

When you want to pass a procedure with variables into a function in Vim script, how do you achieve it?

Let me explain with other language first, show a well-known function map.

Ruby(*1):

a = 10
p [1, 2, 3].map {|i| i + a }
#=> [11, 12, 13]

Python:

a = 10
print map(lambda x: x + a, [1, 2, 3])
#=> [11, 12, 13]

or more strictly

a = 10
def f(x):
    return x + a
print map(f, [1, 2, 3])
#=> [11, 12, 13]

On the other hand, Vim script doesn't have such fancy feature but have normal function and dictionary function. The former is like just a global function, scoped globally or file. The latter is like a method which has the concept of this of self in other languages.

let obj = {'a': 10}
function! obj.f(x)
  return a:x + self.a
endfunction

echo obj.f(1)
"=> 11

Note that the prefix a: means the identifier is an argument of the function.

This feature is compatible with lexical scope lambda except for the aspect of the anonymity. But it is very inconvenient that you always have to declare which variables the dictionary function will use.

The solution is to use a special variable l:.

function! Main()
  let a = 10
  let obj = copy(l:)
  function! obj.f(x)
    return a:x + self.a
  endfunction

  echo obj.f(1)
endfunction

call Main()

There are three notes for using this trick:

l: is only available in a function.
it can cause SEGV if you forget writing copy
a: as well if you use not only local variables

Acknowledge

thinca and tyru taught me this trick for my problem presentation. thinca is using a kind of this trick in his product.

Footnote

The following code also works on Ruby.

a = 10
f = ->(x) { x + 1 }
p [1, 2, 3].map(&f)

Memoized recursive fibonacci in Python

2010-11-12T17:28:00.001-08:00

A slow literal implementation of fibonacci function in Python is like the below:

def fib(n):
    return n if n < 2 else fib(n-2) + fib(n-1)

This is slow but you can make it faster with memoize technique, reducing the order.

__fib_cache = {}
def fib(n):
    if n in __fib_cache:
        return __fib_cache[n]
    else:
        __fib_cache[n] = n if n < 2 else fib(n-2) + fib(n-1)
        return __fib_cache[n]

This is fast, but obviously dirty. Fortunately Python has decorator feature that gives you much better way of writing.

def memoize(f):
    cache = {}
    def decorated_function(*args):
        if args in cache:
            return cache[args]
        else:
            cache[args] = f(*args)
            return cache[args]
    return decorated_function

@memoize
def fib(n):
    return n if n < 2 else fib(n-2) + fib(n-1)

The @memoize decorator is not only for this fib() function but also for general purpose. I referred this page. This is succinct and clean.

But I felt that the definition of memoize(), particularly decorated_function(), was too long. If the else clause consists of single return statement, you may re-write decorated_function() in 1 line.

Takeshi Abe taught me how to make the two lines into one single line.

cache[args] = f(*args)
return cache[args]

is equivalent to

cache.update({args: f(*args)}) or cache[args]

So let here is the final version of fib with memoize:

def memoize(f):
    cache = {}
    return lambda *args: cache[args] if args in cache else cache.update({args: f(*args)}) or cache[args]

@memoize
def fib(n):
    return n if n < 2 else fib(n-2) + fib(n-1)

It's fast, short, succinct, and cool.

Footnote

This approach may be against the Python golden way.

How to make a Unite plugin

2010-11-08T18:46:00.001-08:00

Unite

The hottest Vim plugin these days is unite.vim.

:Unite file command opens a unite dialog that you can specify file names with inputting text.

You can search not only files but also other resources like buffers you've opened. Unite also allows users to create your own "resource" and to define corresponding actions.

Here I'll make a demonstrative unite plugin unite-colorscheme.

colorscheme

Vim, particularly GUI version of Vim implementations like MacVim, has colorscheme that allows you to change the appearance very much. You may change the colorscheme of the Vim just by :colorscheme ujihisa, but it's not trivial to find which colorschemes you already have. (*1)

unite-colorscheme.vim

https://github.com/ujihisa/unite-colorscheme

unite-colorscheme consists of the following two files.

autoload/unite/sources/colorscheme.vim:

let s:unite_source = {
      \ 'name': 'colorscheme',
      \ }

function! s:unite_source.gather_candidates(args, context)
  " [(name, dir)]
  " e.g. [('adaryn', '/Users/ujihisa/.vimbundles/ColorSamplerPack/colors'), ...]
  let colorlist = map(split(globpath(&runtimepath, 'colors/*.vim'), '\n'),
      \'[fnamemodify(v:val, ":t:r"), fnamemodify(v:val, ":h")]')

  return map(colorlist, '{
        \ "word": v:val[0],
        \ "source": "colorscheme",
        \ "kind": "colorscheme",
        \ "action__path": printf("%s/%s.vim", v:val[1], v:val[0]),
        \ "action__directory": v:val[1],
        \ }')
endfunction

function! unite#sources#colorscheme#define()
  return s:unite_source
endfunction

And autoload/unite/kinds/colorscheme.vim:

let s:kind = {
      \ 'name': 'colorscheme',
      \ 'default_action': 'execute',
      \ 'action_table': {},
      \ 'parents': [],
      \ }
let s:kind.action_table.execute = {
      \ 'is_selectable': 1,
      \ }
function! s:kind.action_table.execute.func(candidates)
  if len(a:candidates) != 1
    echo "candidates must be only one"
    return
  endif
  execute "colorscheme" a:candidates[0].word
endfunction

function! unite#kinds#colorscheme#define()
  return s:kind
endfunction

After you installed the plugin, you can search by :Unite colorscheme and set the colorscheme on the Vim just by selecting one colorscheme from the choice.

References and footnote

(*1) neocomplcache.vim supports colorscheme completion.
vim-ref
- only the following two files: kinds/ref.vim and sources/ref.vim
"How to cook a source of Unite (in Japanese)"

The Implementation of Enumerable#map of JRuby

2010-11-07T17:44:00.001-08:00

See src/org/jruby/RubyEnumerable.java.

@JRubyMethod(name = {"map"}, frame = true, compat = CompatVersion.RUBY1_9)
public static IRubyObject map19(ThreadContext context, IRubyObject self, final Block block) {
    return collectCommon19(context, self, block, "map");
}

This looks like defining the map19 method with declaring it's the map method in Ruby, particularly in 1.9 mode. For some reason this is internally collectCommon19() not mapCommon19().

private static IRubyObject collectCommon19(ThreadContext context, IRubyObject self, final Block block, String methodName) {
    final Ruby runtime = context.getRuntime();
    if (block.isGiven()) {
        final RubyArray result = runtime.newArray();

        callEach19(runtime, context, self, new BlockCallback() {
            public IRubyObject call(ThreadContext ctx, IRubyObject[] largs, Block blk) {
                IRubyObject larg = checkArgs(runtime, largs);
                IRubyObject value = block.yield(ctx, larg);
                synchronized (result) {
                    result.append(value);
                }
                return runtime.getNil();
            }
        });
        return result;
    } else {
        return enumeratorize(runtime, self, methodName);
    }
}

This is the definition of collectCommon19. Focus on the then clause of the main if condition block. It's basically calling callEach19() with an instance of BlockCallback, redefining call(). This looks like block-passing style in Ruby.

Focus also on the line just before the callEach19(). The variable result looks like an empty Ruby array for storing values in map process. The definition of the method newArray() is the following.

public RubyArray newArray() {
    return RubyArray.newArray(this);
}

OK. How about RubyArray.newArray().

/** rb_ary_new
 *
 */
public static final RubyArray newArray(final Ruby runtime) {
    return newArray(runtime, ARRAY_DEFAULT_SIZE);
}

Ok, next. Note that the constant ARRAY_DEFAULT_SIZE is an int value.

public static final RubyArray newArray(final Ruby runtime, final int len) {
    RubyArray array = new RubyArray(runtime, len);
    RuntimeHelpers.fillNil(array.values, 0, array.values.length, runtime);
    return array;
}

Looks like [0, ..., 0] with size ARRAY_DEFAULT_SIZE. So you may wonder is it OK to define a longer array if the receiver of map has longer size than ARRAY_DEFAULT_SIZE? I also wonder.

The Problem and Solution of Haml

2010-11-05T16:57:00.001-07:00

When I write an HTML file I always write in Haml first and generate HTML by haml command. Haml is so handy that Haml makes us free from writing closing tags like </p>. The problem in Haml is that it is very difficult to type % very often. The symbol is the second most difficult location (*1) in a keyboard.

The solution is to write the following code on you ~/.vim/ftplugin/haml.vim which let : key as % only in Haml file and when the cursor is on the top of the line.

inoremap <buffer><expr> : <SID>on_the_top() ? '%' : ':'
function! s:on_the_top()
  return col('.') == 1 || getline(line('.'))[0:col('.')] =~ '^\s*$'
endfunction

Note that I swapped : and ;. If you didn't swap them, just to swap your keyboard or the code.

(*1) The most difficult location key is ^ (shift + 6). If you want to make a programming language which is highly difficult to code, use ^ a lot.

List of Vim plugins this computer has

2010-10-31T13:30:00.001-07:00

Most of Vim plugins I'm using is controlled under pathogen until the day VimJolts will be released.

ColorSamplerPack
- A collection of colorschemes. I use zenburn in this collection or mrkn256 not in this collection.
IndentAnything
- Only for JavaScript indentation Support.
blogger.vim
- For blogging. Written by me.
camelcasemotion
- ,w as "go to the next capital character."
ghc_complete
- A neocomplcache plugin for ghc.
gist-vim
- To post the current buffer to gist.
git-vim
- To commit the current file or to view git diff.
hootsuite
- For HootSuite development.
indent-haskell.vim
- An indent file for Haskell.
neocomplcache
- The ultimate auto complete tool.
quickrun
- The ultimate quick running tool.
repl.vim
- To start repl in Ruby, Erlang and Haskell. Written by me.
shadow.vim
- The ultimate file-shadowing tool.
stickykey.vim
- To emulate normal keys as modifier keys.
- I use this for CoffeeScript trick. http://vim-users.jp/2010/07/hack164/
thinca-poslist
- <C-o> can go back at any movements including j.
uj-textile
- ftdetect, ftplugin and syntax file for textile.
- I suppose this plugin is written by someone. I wonder why this dir name contains uj prefix...
unite.vim
- Anything.el
vim-altercmd
- Automatically capitalize a Vim command in cmdwin
vim-coffee-script
- ftdetect, ftplugin, indent and syntax file for CoffeeScript.
vim-markdown
- ftdetect, ftplugin and syntax file for Markdown.
vim-smartchr-0.1.0
- My mappings in vimrc is full of the flavor of smartchr.
- My favorite plugin.
vim-smartword-0.0.2
- An extention of w or e.
vim-surround
- cs([ changes the parenthes into brackets.
vim-textobj-indent-0.0.3
- You can select by indentation.
vim-textobj-user-0.3.8
- You can define your own textobj easily.
vimclojure-2.1.2
- A lot of supports for clojure including ftdetect, ftplugin, index and syntax.
- This has repl support but I've never used it so far.
vimproc
- The :!, :r! and system() substitutor.
- This is used by quickrun, repl.vim, vimshell and .vimrc
vimshell
- A shell for vim.

Syntax files

gas
- for Gnu Assembly language.
haskell.vim
rdoc.vim
- obviously

Obsolete files

This blog post helped me finding some obsolete plugins that I should throw away.

shim
- An interactive Haskell shell in Vim.
- repl.vim is a superset of this plugin.
hints_man2
- ?

Is Hash Comment in SQL?

2010-10-27T18:28:00.001-07:00

A "#" is not the character to start a comment in SQL, but "--" is.

A "#" is the character to the end of the line in MySQL.

If you are a Vimmer and only use MySQL when you edit .sql files, you may want to regard the characters after #. .sql files are regarded as just SQL in Vim, but you can specify them as MySQL. Just to write the following one line on ~/.vim/ftdetect/mysql.vim.

autocmd BufRead,BufNewFile *.sql set filetype=mysql

Reading Git Log In Kindle

2010-10-21T10:02:00.001-07:00

I bought a kindle last week and realized how awesome it was.

By the way most programmers' work begins with reading git-log before they start writing great codes. If you can read it with your kindle, having fantastic coffee, it should be amazing experience.

How to do that

The git log after you git pull with time ascending order is available with the following command (*1).

$ git log HEAD@{1}..HEAD --reverse

You can read the diffs as well, with -u option.

About coloring. Vim has diff syntax coloring system and can generate an html file with the color.

$ git log -u HEAD@{1}..HEAD --reverse | vim - -c 'TOhtml' -c 'w | qa!'

This generates Untitled.html.

$ open ./Untitled.html

And

Open Print dialog in your safari
Save PDF
Send the file to Kindle

That's it. Enjoy!

References

A Java Library Looks Like A JRuby Library

2010-10-13T21:40:00.001-07:00

Problem

In the article I wrote, Introduction to JRuby - Bidirectional Communication, I described how to define a class in Java and call it in JRuby. The class written in Java is called by JRuby in a different way to a library written in pure ruby.

For example, assuming now you are making a Ruby library that has a class and a singleton method Hamburger.arrayze. If it's in Ruby you can write a single rb file hamburger.rb and write the following code.

hamburger.rb:

class Hamburger
  def self.arrayze(x)
    ['begin', x, 'end']
  end
end

and call it from another rb file:

require 'hamburger'
p Hamburger.arrayze('hello')
#=> ["begin", "hello", "end"]

You also can implement Hamburger class in pure Java.

Hamburger.java:

class Hamburger {
  static public String[] arrayze(String x) {
    String[] memo = {"begin", x, "end"};
    return memo;
  }
}

Compile it into a jar file.

$ javac Hamburger.java
$ jar cf hamburger.jar Hamburger.class

Then call the library from an rb file.

require 'hamburger.jar'
include_class Java::Hamburger

p Hamburger.arrayze('hello').to_a
#=> ["begin", "hello", "end"]

The result ran on JRuby is exactly same to the previous one, but the code is very different. You cannot delete .jar extension in require method, abbreviate the include_class declaration, or .to_a to convert Java class Object to Ruby Array of String.

Solution 1

First we should not to return a Java object but to return a Ruby array.

Git-SVN Low-Risk Practice

2010-10-13T20:16:00.001-07:00

Assumptions

You are supposed to work on a svn branch topic.
You hate svn.
You want to follow changes of svn trunk.
You hate the merging in svn.
You already checked out the svn repository with git-svn-clone with stdlayout option.

Wrong way

First you make a local git branch topic that'll sync with svn branch topic.

$ git checkout -b topic topic

Then you work on the branch, including git-commit a lot. Before you go home you push the changes into the main svn repository by git-svn-dcommit.

You sometimes have to see the changes in trunk.

$ git checkout master
$ git svn rebase

And merges the changes into your topic branch.

$ git checkout topic
$ git merge --no-ff master

You know that merge without --no-ff option breaks git-svn system so you did --no-ff.

Finally you finished all your work on the topic branch and tried to merge the changes into trunk.

$ git checkout master
$ git merge --no-ff topic

That causes a lot of conflicts that you have to resolve manually.

Right way

The "Wrong way" example had two failures.

ambiguous name of topic
merge from trunk to topic

The first one is not fatal but annoying. Every time you run a git command with specifying a branch name shows a warning message that your local git branch name is ambiguous. You should have make a local git branch in which name is different to the svn branch.

$ git checkout -b tpc topic

Next. You are not supposed to merge changes in trunk to topic branch that will be merged to trunk. That causes a lot of conflicts.

The solution is that you create another local git branch that doesn't sync svn branch directly.

$ git checkout -b t tpc

Instead of trying to merge from master to t, try to rebase from master to t. Regard the tpc branch for very temporary one.

$ git checkout master
$ git svn rebase
$ git checkout t
$ git rebase master

$ git checkout tpc
$ git svn dcommit
$ svn delete `git svn info --url`
$ git checkout t
$ git branch -d -r topic
$ git branch -D tpc
$ git svn branch topic
$ git checkout -b tpc topic

Note that this approach make the svn repository log messy. Be resigned; that's the subversion.

RSpec for Java

2010-09-07T19:48:00.001-07:00

I think the reasons why people are using Java are in the following list.

The product must run on JVM
You are fixing existing code that is written in Java
You or your co-workers love Java

Java has some testing frameworks written in Java, but it's difficult to implement something like RSpec due to the limitation of Java syntax. But actually you don't have to use a testing framework in Java but can use RSpec on JRuby, even though the product is not using JRuby.

A minimum example

Hello.java

public class Hello {
  public String world() {
    return "world!";
  }
}

hello_spec.rb

require 'java'
java_import 'Hello'

describe 'Hello' do
  it 'world' do
    Hello.new.world.should == 'world!'
  end
end

Compile Hello.java and run RSpec command.

$ javac Hello.java
$ jruby -S spec hello_spec.rb

This works.

If you didn't install RSpec yet, you can install it with the following command.

$ jruby -S gem install rspec

A bit complex example

This approach is powerful; you can use a Java bytecode which was compiled with some special parameters like -target. For example, J2ME programs needs to be compiled as Java 1.4 and needs a jar file as well. You still can spec such program.

A.java

import javax.microedition.midlet.MIDlet;

class A extends MIDlet {
  public void destroyApp(boolean unconditional) {
    notifyDestroyed();
  }
  protected void pauseApp() {
  }
  protected void startApp() {
  }
  static public String f() {
    return "sajdfksadfsd";
  }
}

a_spec.rb

require 'java'
java_import 'A'

describe 'A.f' do
  it 'is on java' do
    RUBY_DESCRIPTION.should be_include 'jruby'
  end

  it 'is a strange string' do
    A.f.should == 'sajdfksadfsd'
  end
end

run

$ javac -target 1.4 -source 1.4 -g:none -bootclasspath the-jar-file.jar A.java
$ jruby -J-cp the-jar-file.jar:. -S ~/git/jruby/bin/spec a_spec.rb

It works.

Shows Message Immediately in Haskell

2010-09-05T19:32:00.001-07:00

You've seen such a prompt very often.

OK? [y/n]

Your cursor is just after the prompt with 1 space. If you type y the process will continue and if you type any other characters, not only n, the process will stop. This is easy to implement in Ruby.

print "OK? [y/n] "
if gets.chars.first == 'y'
  puts 'go go go'
else
  puts 'nooo'
end

It seems to be easy for Haskell beginners to implement it in Haskell like the following code.

main = do putStr "OK? [y/n] "
          s <- getLine
          case (head s) of
               'y' -> putStrLn "Nice"
               otherwise -> putStrLn "omg"

But it doesn't work appropriately due to stdout buffering. Haskell has IO library that can control buffering. Add hFlush stdout after importing the library.

import IO (hFlush, stdout)

main = do putStr "OK? [y/n] "
          hFlush stdout
          s <- getLine
          case (head s) of
               'y' -> putStrLn "Nice"
               otherwise -> putStrLn "omg"

This works.

Regular Expression Benchmarks

2010-08-16T20:38:00.001-07:00

Most modern programming languages have regular expression engines. Here I'll compare the speed of each regular expression engines with long text matching which has backtracking. I used Ruby, Perl and Python for this examination.

A time-consuming regular expression example

/(aa)*aa/ =~ 'aa' needs backtracking. The longer the length is, the longer it would take. Let m as the length of the characters and n as the number of repeat. For example the case when n = 2 and m = 3 is

/(aa)*aa(bb)*bb(cc)*cc/ =~ 'aabbcc'

Codes

Ruby:

def time
  t = Time.now
  yield
  Time.now - t
end

def ryutorion(m, n)
  chars = (0...n).map {|i| ('a'.ord + i % 26).chr }
  regex = Regexp.new chars.map {|c| "(#{c*m})*#{c*m}" }.join
  str = chars.map {|c| c*m }.join
  time { regex =~ str }
end

Python:

import re
import time

def timeof(f):
    t = time.time()
    f()
    return time.time() - t

def ryutorion(m, n):
    chars = [ chr(ord('a') + i % 26) for i in range(n) ]
    regex = re.compile(''.join([ '(' + c*m + ')*' + c*m for c in chars ]))
    str = ''.join([ c*m for c in chars ])
    return timeof(lambda: regex.match(str))

Perl:

use warnings;
use strict;
use Time::HiRes qw(gettimeofday tv_interval);

sub gen_regex {
    my ($m, $n) = @_;

    join "", map { 
        my $s = chr(ord('a') + ($_ % 26)) x $m;
        "($s)*$s"
    } (0 .. $n - 1);
}

sub gen_input {
    my ($m, $n) = @_;

    join "", map { chr(ord('a') + ($_ % 26)) x $m } (0 .. $n - 1);
}

my $r = gen_regex 1, 1;
my $i = gen_input 1, 1;
$r = qr/$r/;
my $ts = [gettimeofday];
my $te = [gettimeofday] if($i =~ /$r/);
my $elapsed = tv_interval $ts, $te;
print "$elapsed\n";

for my $N (10, 20) {
    for my $M (10000, 20000) {
        $r = gen_regex $M, $N;
        $i = gen_input $M, $N;
        $r = qr/$r/;
        $ts = [gettimeofday];
        $te = [gettimeofday] if($i =~ /$r/);
        $elapsed = tv_interval $ts, $te;
        print "$elapsed\n";
    }
}

This perl example was written by Masahiro Kimoto.

Result

shorter test: n = {10000, 20000} = m = {10, 20}

Ruby

p ryutorion(10000, 10) #=> 0.000278
p ryutorion(20000, 10) #=> 0.000698
p ryutorion(10000, 20) #=> 0.000538
p ryutorion(20000, 20) #=> 0.001026

Python

print ryutorion(10000, 10) #=> 0.00126099586487
print ryutorion(20000, 10) #=> 0.00168895721436
print ryutorion(10000, 20) #=> 0.00180792808533
print ryutorion(20000, 20) #=> 0.00338697433472

Perl

($M = 10000, $N = 10) #=> 0.000066
($M = 20000, $N = 10) #=> 0.000185
($M = 10000, $N = 20) #=> 0.000163
($M = 20000, $N = 20) #=> 0.000294

longer test: n = {10000, 20000} = m = {100, 200}

Ruby (ruby 1.9.2)
```
0.002721
0.00672
0.005441
0.013364
```

Python (python 2.7)

Traceback (most recent call last):
  File "a.py", line 16, in <module>
    print ryutorion(10000, 100)
  File "a.py", line 11, in ryutorion
    regex = re.compile(''.join([ '(' + c*m + ')*' + c*m for c in chars ]))
  File "/usr/local/Cellar/python/2.7/lib/python2.7/re.py", line 190, in compile
    return _compile(pattern, flags)
  File "/usr/local/Cellar/python/2.7/lib/python2.7/re.py", line 243, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/local/Cellar/python/2.7/lib/python2.7/sre_compile.py", line 511, in compile
    "sorry, but this version only supports 100 named groups"
AssertionError: sorry, but this version only supports 100 named groups

Perl (perl 5.10.0)
```
0.000981
0.00269
0.004181
0.005484
```

Summary

Perl is the strongest. Use Perl.

Haskell liftM practice

2010-08-15T13:37:00.001-07:00

with lists

I was trying to remember how to use Haskell. This blog post focuses only on Control.Monad.liftM function.

The type of liftM is (Monad m) => (a1 -> r) -> m a1 -> m r which evaluates the second argument with the first argument as if it's in do block.

Let me explain that with the following example. You need the list of the values which each of them are three times bigger than the original list [1, 2, 3]. The most straightforward way in Haskell is to use list comprehension.

[ x * 3 | x <- [1, 2, 3]]

List Comprehension is just a syntactic sugar of List Monad. The code is equivalent to the following code.

do x <- [1, 2, 3]
   return $ x * 3

Do notation is just a syntactic sugar of List Monad as well.

[1, 2, 3] >>= \x -> return (x * 3)

You also can make the code simpler with point free style.

[1, 2, 3] >>= return . (* 3)

Now it's time to use liftM. You can write the code with liftM in the following way.

liftM (* 3) [1, 2, 3]

This is the simplest. Note that you have to declare import Control.Monad to use liftM.

with Parsec

The code below is from Write Yourself a Scheme in 48 Hours.

parseNumber = liftM (Number . read) $ many1 digit

I rewrite it without liftM.

parseNumber = do x <- many1 digit
                 return $ Number $ read x

parseNumber = do x <- many1 digit
                 return $ (Number . read) x

parseNumber = many1 digit >>= \x -> return $ (Number . read) x

parseNumber = many1 digit >>= \x -> (return . Number . read) x

parseNumber = many1 digit >>= return $ Number . read

summary

Using liftM can make code simpler and easier to understand with its natural order of arguments.

The most difficult thing for using liftM is to write the correct spelling of liftM. I cannot remember how many times I wrote listM instead of liftM. It is very difficult and fatal issue.

Combining Java, Clojure and JRuby

2010-07-15T21:14:00.001-07:00

JRuby can access classes implemented in pure Java code. Clojure can generate a class file. Therefore it's straightforward for JRuby to access classes implemented in Clojure.

First I define a class with a method in plain Java.

MrJava.java:

If you kill pid0, will the internal fork be automatically killed?

Try the previous code. This doesn't show 'bah!' message. It seems to be true that internal fork will be automatically killed. But it's wrong. This code doesn't show 'bah!' message just because the internal fork didn't started yet when Process.kill was called.

Try to wait until both blocks are certainly for forked.

This code shows 'bah!' message after the main routine finished!

So, let's think about how to kill the internal process.

The value of pid1 is not available in the outer main routine
An external command ps is not available in Windows
Usually the process ID pid0 and pid1 are sequential, but depending on it is dangerous

Here is the solution of it:

I added a line of at_exit. This is cumbersome a little bit, but safe and easy.

With Spawn

The previous discussion was actually for getting ready. Windows doesn't have fork. We can use spawn instead of fork in most cases.

This succeeded!

One More Thing

When you have to fork a ruby process with spawn, for example in case to rewrite fork to spawn, how do you write?

fork { p Dir.pwd }

spawn 'ruby', '-e', 'p Dir.pwd'

is not correct. The ruby that booted the code itself is not always 'ruby'. It can be ruby19, jruby, rbx or other rubies. In this case, your the following approach.

require 'rbconfig'
spawn RbConfig::CONFIG['ruby_install_name'] + RbConfig::CONFIG['EXEEXT'], '-e', 'p Dir.pwd'

Compatible Array#flatten

2010-03-11T12:58:00.001-08:00

Array#flatten with depth argument is useful.

[[1], [2], [[3], 4]].flatten(1)
#=> [1, 2, [3], 4]

That's been available since ruby 1.8.7, but older ruby such as ruby 1.8.6 doesn't support it.

Here is the pure-ruby implementation of Array#flatten. Use it in case.

I used it for my library spawn-for-legacy to support ruby 1.8.6.

Rails programmers don't need this code. Rails now only supports ruby newer than 1.8.7.

How To Run An External Command Asynchronously

2010-03-07T00:11:00.001-08:00

This issue looks easy, but is actually complex.

On shell, you may run a command asynchronously just by adding & at the end of command.

$ sleep 10 &
[1] 8048

You can obtain the process ID, so you can stop the command.

$ kill 8048

How can we run an external command on Ruby?

system()

The easiest way to run an external command is system().

system 'sleep 10 &'

The return value of system() is the exit code. In this case, the system() returns true immediately after that ran, but it doesn't return process ID. To obtain the process ID, you can use $?.

system 'sleep 10 &'
p $? #=> 8050

But unfortunatelly the $? is not the processs ID of sleep, but sh -c sleep 10. There is a cussion. You cannot obtain the process ID of sleep directly.

One more problem: the notation is available only in UNIX based platform.

system() without shell

system 'sleep 10' uses shell but system 'sleep', '10' doesn't use shell. Separate the arguments. But system 'sleep', '10', '&' is not available.

Thread

To emulate '&' feature, how about using Ruby's asynchronous features? Thread is the easiest way of that.

t = Thread.start do
  system 'sleep', '10'
end
t.kill

Looks good, but it doesn't work. Thread certainly will stop, but the external command booted by the thread will still alive.

exec()

How about using another way of booting an external command? There is exec(), which roughly means the combination of system() and exit().

exec 'ls'

roughly means

system 'ls'
exit

So you think that the following code will work well.

t = Thread.start do
  exec 'sleep', '10'
end
t.kill

Unfortunatelly it doesn't work. You cannot use exec() in a child thread.

fork()

Look for another way of supporting asynchronous feature on Ruby. fork(). fork() copies the Ruby process itself.

pid = fork do
  exec 'sleep', '10'
end
Process.kill pid

It works! fork() is awesome!

But there is a bad news. Thread works on all platforms, but fork works on only some platforms. I'm sorry Windows users and NetBSD users.

spawn()

spawn = fork + exec. spawn() works on all platforms! It's perfect! But... ruby 1.8 doesn't have spawn()...

Summary

Ruby 1.9 + UNIX: Use fork and exec, or use spawn.
Ruby 1.8 + UNIX: Use fork and exec.
Ruby 1.9 + Windows: Use spawn.
Ruby 1.8 + Windows: Use other OS, other Ruby, or consider using this library

How To Avoid The Worst Case

2010-03-06T13:17:00.001-08:00

I broke an expensive unopened wine bottle. The white wine was spilt all over my kitchen.

Yesterday I bought a wine bottle and a box of beer at a liquor store which is located far from my home. I bought them because they were on sale. To save my money, I came home on my foot, saving $1.75. Today I ate lunch and drunk a sip of beer. After the lunch, I was trying to get a snack. There was the wine bottle in front of the snack. My elbow hit the bottle, and broke it. The delicious expensive wine has gone. $14 and the heavy work has gone.

While I was wiping the floor with smelling the wine, I was thinking what was the cause and how should I do after that. This incident was happened without any other people except for me. I'm the true culprit. So, how can I avoid such tragedy in my future?

The causes are the following two. The fact I was drunk a little bit and the location of wine was not safe a little bit. Both of them are not crucial. But the incident certainly happened.

This issue can be summarized as that we should assume anything for people who are in bad condition. People often become bad because of drowsiness, alcohol, anger and depression. Assume the condition. Never put glass products on the edge, or the place people can touch easily.

All About Spawn

2010-03-05T15:38:00.001-08:00

Spawn = Fork + Exec, but works also on platforms which don't support fork.

The usage of spawn is often said to be cryptic. Here I'll describe common cases.

system('ls')

pid = spawn(['ls', 'ls'])
Process.wait(pid)

system('ls .')

pid = spawn('ls', '.')
Process.wait(pid)

system('ls . &')
```
pid = spawn('ls', '.')
```

system('ls . > a.txt')

pid = spawn('ls', '.', :out => 'a.txt')
Process.wait(pid)

system('ls . >> a.txt')

pid = spawn('ls', '.', :out => ['a.txt', 'a'])
Process.wait(pid)

system('ls . >& a.txt')

pid = spawn('ls', '.', [:out, :err] => ['a.txt', 'w'])
Process.wait(pid)

IO.popen('cat a.txt') {|io| p io.read

i, o = IO.pipe
spawn('cat a.txt', :out => o)
o.close
p i.read

system('make all &')
```
spawn('make', 'all)
```
Dir.chdir('a') { system 'make all &' }
```
spawn('make', 'all', :chdir => 'a')
```
Passing ENV:
- Shell: $ AAA=1 make all &
- Ruby: ENV['AAA'] = '1'; system('make all &')
- Ruby with Spawn: spawn({'AAA' => '1'}, 'make', 'all')

Further information can be available in process.c in ruby trunk. Here the documentation from the file:

LLVM Workshop at Online.sg

2010-02-25T21:41:00.001-08:00

I gave a lecture about LLVM.

The slides are available here: http://github.com/ujihisa/onsg9/blob/master/slides.md

I emphasized the following points. The optimizer of LLVM is awesome. I explained it with giving an example code.

First, I generated a very long helloworld code. To generate lavish code, I used a Brainf**k compiler. The helloworld written in Brainf**k doesn't have something like String literal, so it's natural that the code is longer than necessary, therefore the generated LLVM assembly language code would be lavish as well.

$ bfc.rb --llvm helloworld.bf > helloworld.ll

The line number of the automatically generated LLVM assembly language code was 2714. Here let me optimize the code.

$ llvms helloworld.ll -o a.bc
$ opt -O3 a.bc -o a2.bc
$ opt -O3 a2.bc -o a3.bc
$ llvm-dis a3.bc -o > a.ll

The optimized code, a.ll, is semantically equivalent to the original helloworld.ll, though the lines of code is only 25.

All attendees were surprized about the result.

Redirecting STDOUT

2010-02-25T13:25:00.001-08:00

In writing spec in Ruby, you would often meet a situation that you have to reduce outputting messages to STDOUT.

def f
  p 123
end

describe 'f' do
  it 'outputs "123\n" to stdout' do
    ...
  end
end

How do you write it? Usually I assign a stringio object to $stdout. I guess many people use this approach.

def f
  p 123
end

# spec
require 'stringio'
results = $stdout = StringIO.new
f()
results.close_write
results.rewind
$stdout = STDOUT
p results.read #=> "123\n"

I started having an idea that IO#reopen may be a better solution of it. The code previously given looks obviously dirty.

Finally I figured out the new approach was also dirty.

def f
  p 123
end

require 'tempfile'
results = Tempfile.new('a').path
a = STDOUT.dup
STDOUT.reopen(results, 'w')
f()
STDOUT.reopen(a)
p File.read(results)

STDOUT.reopen doesn't accept an IO object, but only accepts a Filename.

capture_io of minitest

Eric Hodel and Ryan Davis mentioned about capture_io in minitest. It uses the former approache.

def capture_io
  require 'stringio'

  orig_stdout, orig_stderr         = $stdout, $stderr
  captured_stdout, captured_stderr = StringIO.new, StringIO.new
  $stdout, $stderr                 = captured_stdout, captured_stderr

  yield

  return captured_stdout.string, captured_stderr.string
ensure
  $stdout = orig_stdout
  $stderr = orig_stderr
end

Reference

http://www.ruby-forum.com/topic/187367

Eric Hodel++

Multi Process Programming With DRb

2010-02-17T12:22:00.001-08:00

Usually Ruby's thread works well, but when you have to use thread-unsafe features like Dir.chdir, you need to use fork. How can you wait for the children processes certainly finish the given tasks? The answer is to use DRb.

I'll show an example. Prepare the following two files and run them:

server.rb

require 'drb'
x = Array.new(10, false)
def x.work(i)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end
DRb.start_service 'druby://localhost:1234', x

puts DRb.uri
DRb.thread.join

client.rb

require 'drb'
x = DRbObject.new nil, 'druby://localhost:1234'
10.times do |i|
  fork { x.work(i) }
end

def wait_until
  loop do
    yield and return
    sleep 0.1
  end
end

t = Time.now
wait_until { x.finish? }
puts "#{Time.now - t}sec."

run!

$ ruby server.rb &
$ ruby client.rb
...

The server has 10 small virtual tasks which can be finished within 1 second. The x.work(i) is the definition. Because of that, ideally the whole process finish maximally in 1sec, but practically because of reasons such as the definition os wait_until, the whole process will finish maximally about 1.1sec.

require 'drb'
n = 10

# Server
x = Array.new(n, false)
def x.work(i)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end

fork do
  DRb.start_service 'druby://localhost:1234', x
  puts DRb.uri
  DRb.thread.join
end

# Client
def wait_until
  loop do
    yield and return
    sleep 0.1
  end
end

x = DRbObject.new nil, 'druby://localhost:1234'
sleep 0.2 # FIXME

n.times do |i|
  fork { x.work(i) }
end

t = Time.now
wait_until { x.finish? }
puts "#{Time.now - t}sec."

I think you noticed that there is a dirty code in it. Yes, "sleep 0.2 # FIXME" is. I tried to find how to wait until the DRbObject certainly connected, but I couldn't find any good solution.

Without `wait_until`

I wrote the process at exit in the method in the server x.work in order not to use wait_until.

require 'drb'
n = 10

# Server
x = Array.new(n, false)
def x.work(i, t)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true

  if finish?
    puts "Total: #{Time.now - t}sec."
  end
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end

fork do
  DRb.start_service 'druby://localhost:1234', x
  puts DRb.uri
  DRb.thread.join
end

# Client
x = DRbObject.new nil, 'druby://localhost:1234'
sleep 0.2 # FIXME

t = Time.now
n.times do |i|
  fork do
    x.work(i, t)
  end
end

At first I used block passing style, but later I found that it's impossible to pass block through druby.

Or

Use Rinda which is a higher layer library using DRb.

How To Save Both Standard And Error Logs By Some Processes On A Same Log File

2010-02-14T16:30:00.001-08:00

Problem

Assuming that you need to save both standard and error logs by two processes on a same file.

$ command1
mesg...
error...
mesg...
....

$ command2
mesg...
mesg...
mesg...
error..
....

How to do that? The following code looks good, but it doesn't work as you want.

$ (command1 >& log.txt) &
$ (command2 >& log.txt) &

You can find only the logs of command2.

How about trying this?

$ (command1 2>&1 >> log.txt) &
$ (command2 2>&1 >> log.txt) &

It is better, but it doesn't work as you want as well. The result is not chronically ordered because of buffering.

Solution

Do the following code:

$ (command1 2>&1 | cat >> log.txt) &
$ (command2 2>&1 | cat >> log.txt) &

It's tricky. The part of | cat >> means not to buffer the outputs.

What I Made

2010-02-11T18:23:00.001-08:00

I need more effort to distribute it.

Introduction to Text Editors

2010-02-07T10:38:00.001-08:00

If there were such a class titled "Introduction to Text Editors", I would be delighted to attend it.

Here is just my imagination.

What is Text
- Text in Natural Languages
- Text in Programming Languages
What is Edit
- Open, save and buffer
- Insert, delete and motion
- Modes: view, insert and command
Operation and motion
- Operations: delete, correct, yank and user-defined operations
- Motions: word, line and paragraph
User interface
- Font, size, color and location
- Multiple windows
- Keyboard and other input devices
Programming aids
- Syntax highlighting
- Indentation
- Execute the code
- Hints
- Tagjump
- Handle related files
Completion
- Dictionary-based completion
- Buffer-based completion
- Omni completion
- Mixed completion
Customization
- Any behavior is customizable
- Key mapping
- Plugin

The two major text editors, Vim and Emacs, have their own terms. I'd like to abstract them and give appropriate names.

How To Check If You Have The Command

2010-02-05T21:56:00.001-08:00

For example, to assume you have the command svn and you don't have the command svn1:

which svn >&/dev/null
if [ $? == 0 ]; then
  echo You have svn
fi

which svn1 >&/dev/null
if [ $? == 1 ]; then
  echo You don\'t have svn1
fi

The Right Way of Running External Commands on Vim

2010-02-03T11:19:00.001-08:00

Today I was struggled with one of the worst behavior of Vim script.

Try the following code on your Vim.

:!echo "#!"

Assuming you know that the command :! is to run the given command on your shell. This looks equivalent to

$ echo "#!"

on your terminal.

But Vim will show a cryptic error. That's because both # and ! are special letters for :! command.

! is the command previously ran. This is for the pseudo command :!! which means that running the same command again.
# is the filename previously you edited.
Similar in %

They may be useful for using Vim, but they make Vim script programmers crazy.

Gmail In Other Fonts

2010-01-06T19:14:00.001-08:00

I use gmail. Gmail supports changing the layout. Gmail doesn't support changing the font family of messages. The default font family is Arial, but today I preferred to use Georgia instead. I tried to fix the font family with JavaScript, but it was totally tough.

My first code:

javascript: var a = document.getElementsByTagName('iframe'); for (var i in a) { var b = a[i].contentWindow.document.getElementsByTagName('div'); for (var j in b) { b[j].setAttribute && b[j].setAttribute('style', 'font-family: Georgia'); } }

It changes all texts in all <div>s, but the layout will be completely broken. The content of an email appears on the bottom of the browswer.

My second code:

javascript: var a = document.getElementsByTagName('iframe'); for (var i in a) { var b = a[i].contentWindow.document.getElementsByClassName('ii gt'); b.item() && b.item().setAttribute('style', 'font-family: Georgia'); }

It certainly changes the font of the content of an email, but when you moved to other messages, the font will back to the default ones.

Finally, my friend Hamachiya2 succeeded in writing the code of it.

javascript:(function(){ function cssr(doc, sel, dec) { var sheets=doc.styleSheets; if (sheets.length) { var tSheet=sheets[sheets.length-1]; tSheet.insertRule(sel+"{"+dec+"}",tSheet.cssRules.length); } }; var ifr=document.getElementById('canvas_frame'); cssr(ifr.contentWindow.document, '.gs', 'font-family: Georgia'); })()

Bookmarklet is here: ujihisa. Drag it to your bookmark bar.

To extend it for ease,

(function(){
  function cssr(doc, sel, dec) {
    var sheets=doc.styleSheets;
    if (sheets.length) {
      var tSheet=sheets[sheets.length-1];
      tSheet.insertRule(sel+"{"+dec+"}",tSheet.cssRules.length);
    }
  };
  var ifr=document.getElementById('canvas_frame');
  cssr(ifr.contentWindow.document, '.gs', 'font-family: Georgia');
})()

This works perfectly. Use it and bless Hamachiya2!

before

after

Knead Programming Language

2010-01-03T14:12:00.001-08:00

interesting

Rabbit On Mac OS X

2010-01-01T16:13:00.001-08:00

To use rabbit the presentation software written in Ruby, it is necessary to install gtk2.

$ sudo port install gtk2

will never be finished. I gave up the installation and typed <C-c>...

Trying Out Bundler In 1 Minute

2009-12-30T13:30:00.001-08:00

Let's write an application which uses the gem library "g" without installing it globally.

Mise En Place

Install bundler to make the application easily.

gem install bundler

And if you already have g, uninstall it to make sure the application you'll make doesn't use the globally installed g.

gem uninstall g

Getting Started

$ mkdir ggg; cd ggg
$ vim Gemfile
gem 'g'
gem 'ruby-growl'

$ gem bundle
$ vim app.rb
require 'vendor/gems/environment'
require 'g'
g 'success!!!'

$ ruby app.rb

That's all!

For your information:

$ tree
.
|-- Gemfile
|-- app.rb
|-- bin
|   `-- growl
`-- vendor
    `-- gems
        |-- cache
        |   |-- g-1.3.0.gem
        |   `-- ruby-growl-1.0.1.gem
        |-- doc
        |-- environment.rb
        |-- gems
        |   |-- g-1.3.0
        |   |   |-- README.markdown
        |   |   |-- Rakefile
        |   |   |-- VERSION
        |   |   |-- g.gemspec
        |   |   |-- lib
        |   |   |   `-- g.rb
        |   |   `-- spec
        |   |       `-- g_spec.rb
        |   `-- ruby-growl-1.0.1
        |       |-- LICENSE
        |       |-- Manifest.txt
        |       |-- Rakefile
        |       |-- bin
        |       |   `-- growl
        |       |-- lib
        |       |   `-- ruby-growl.rb
        |       `-- test
        |           `-- test_ruby-growl.rb
        `-- specifications
            |-- g-1.3.0.gemspec
            `-- ruby-growl-1.0.1.gemspec

14 directories, 20 files

Usually Something, But If...

2009-12-28T13:15:00.001-08:00

Which do you prefer to write in Ruby?

if boundary_condition
  code_for_the_extreme_case
else
  code_for_the_typical_case
  ...
end

unless boundary_condition # `if !boundary_condition` as well.
  code_for_the_typical_case
  ...
else
  code_for_the_extreme_case
end

In such cases, first I try to use guards. This is straightforward.

return code_for_the_extreme_case if boundary_condition
code_for_the_typical_case
...

But sometimes I cannot use such notation in cases where not to use return or break.

Solution

I made a DSL for this problem.

class UsuallyPending
  instance_methods.map {|i| i.to_s }.
    reject {|i| /__/ =~ i }.
    each {|m| undef_method m }

  def initialize(b1)
    @b1 = b1
  end

  def but_if(cond, &b2)
    if cond
      b2.call
    else
      @b1.call
    end
  end
end

def usually(&b1)
  UsuallyPending.new(b1)
end

usually do
  p ARGV
  p 'hello!'
end.but_if ARGV.empty? do
  p 'Give me arguments!'
end

This is a straightforward expansion of postpositive if with block instead of a value.

BFC: Brainf\\k Compilers

2009-12-27T14:24:00.001-08:00

Today I released BFC 1.0!

BFC: Brainf**k Compilers http://github.com/ujihisa/bfc

bfc.rb is a compiler written in Ruby, which can compile BF code to Ruby, C, Haskell, Scheme and LLVM.

USAGE OF BFC

$ ./bfc.rb --help
$ ./bfc.rb [-v|--version]

$ ./bfc.rb [-r|--ruby]    helloworld.bf > helloworld.rb
$ ./bfc.rb [-c|--c]       helloworld.bf > helloworld.c
$ ./bfc.rb [-h|--haskell] helloworld.bf > helloworld.hs
$ ./bfc.rb [-l|--llvm]    helloworld.bf > helloworld.ll
$ ./bfc.rb [-s|--scheme]  helloworld.bf > helloworld.scm

$ cat helloworld.bf | ./bfc.rb --ruby
$ ./bfc.rb [-r|--ruby|-c|--c|-h|--haskell|-l|--llvm] helloworld.bf --run
$ ./bfc.rb [-c|--c] helloworld.bf --without-while > helloworld.c
$ spec ./bfc.rb

THE BRAINF**K LANGUAGE

According to Wikipedia, the programming language Brainf**k has the following 8 tokens that each have semantics. Here is the equivalent transformation from Brainf**k to C.

The bfc.rb converts BF codes to each languages mostly based on the table.

C Translation Table in bfc.rb:

',' => '*h=getchar();',
'.' => 'putchar(*h);',
'-' => '--*h;',
'+' => '++*h;',
'<' => '--h;',
'>' => '++h;',
'[' => 'while(*h){',
']' => '}'

Ruby Translation Table in bfc.rb:

',' => 'a[i]=STDIN.getc.ord',
'.' => 'STDOUT.putc(a[i])',
'-' => 'a[i]-=1',
'+' => 'a[i]+=1',
'<' => 'i-=1',
'>' => 'i+=1',
'[' => 'while a[i]!=0',
']' => 'end'

They are straightforward enough not to be explained the detail.

In the same way, we can write translation tables for most programming languages except special languages including Haskell and Assembly languages.

TRANSLATING TO HASKELL

Translating BF to Haskell needs two tricks. Haskell was difficult to handle BF because:

Variables in Haskell are not allowed to be re-assigned
- ++h is impossible
There's no feature like while statement

So I used IO Monad with biding same-name variables, and defined while function.

Haskell Translation Table in bfc.rb:

',' => 'tmp <- getChar; h <- return $ update (\_ -> ord tmp) i h;',
'.' => 'putChar $ chr $ h !! i;',
'-' => 'h <- return $ update (subtract 1) i h;',
'+' => 'h <- return $ update (+ 1) i h;',
'<' => 'i <- return $ i - 1;',
'>' => 'i <- return $ i + 1;',
'[' => '(h, i) <- while (\(h, i) -> (h !! i) /= 0) (\(h, i) -> do {',
']' => 'return (h, i);}) (h, i);'

And the definition of while is:

while cond action x
 cond x    = action x >>= while cond action
 otherwise = return x

This is short, but can handle loop with changing the value with larger scope like C's.

TRANSLATING TO C WITHOUT WHILE STATEMENTS

Unlike the effort on Haskell, it is impossible to write simple translation table for C when I can use only goto for control flows instead of while statements. So I made the compile to have label counters to make labels for goto a lot.

Excerpt from bfc.c:

when ','; '*h=getchar();'
when '.'; 'putchar(*h);'
when '-'; '--*h;'
when '+'; '++*h;'
when '<'; '--h;'
when '>'; '++h;'
when '['; "do#{counter += 1}:"
when ']'
  "if (*h != 0) goto do#{counter}; else goto end#{counter};" <<
  "end#{counter}:"
end

TRANSLATING TO LLVM

LLVM Assembly language is similar to Haskell to the extent of the prohibition of re-assignments, and not similar to Haskell to the extend of having do syntax for Monad. So I decided to use pointers to store values. Also, LLVM needs many temporary variables which cannot be re-assigned, so I used counters again to use temporary constants.

The translation table with counters is too big to paste here, so I'll just show the definition of '+' which means '++h' in C.

when '+'
  a = tc += 1; b = tc += 1; c = tc += 1; d = tc += 1
  "%tmp#{a} = load i32* %i, align 4\n" <<
  "%tmp#{b} = getelementptr [1024 x i8]* %h, i32 0, i32 %tmp#{a}\n" <<
  "%tmp#{c} = load i8* %tmp#{b}, align 1\n" <<
  "%tmp#{d} = add i8 1, %tmp#{c}\n" <<
  "store i8 %tmp#{d}, i8* %tmp#{b}, align 1\n"

(where tc is the abbreviation of tmp counter.)

One more thing. LLVM is famous for its aggressive optimizations. For example, the result of the conversion from helloworld.bf to LLVM Assembly Language is very long.

$ ./bfc.rb --llvm ./helloworld.bf | wc -l
2842

But once you optimize the assembly by opt command of LLVM, the line of code will become shorter and more succinct.

SUMMARY

BFC supports compiling BF to the following language.

Ruby
C
Haskell
LLVM Assembly Language
Scheme

In some languages it was easy to write the translator, but Haskell and LLVM was tough for me.

If I have a plenty of time, I'd like to try these challenges:

Compiling to Erlang
Compiling to IA-32 Assembly Language
Compiling to LLVM Bitcode
More Spec!
Benchmark Suite

Anyway, I recommend you to take a look at the BFC. Enjoy!

LLVM For Starters

2009-12-26T12:09:00.001-08:00

Installation of LLVM Compiler and Runtime

See the previous post.

Overview of LLVM

To write a helloworld application, you can choose a path where to start. The typical path is,

Write a code in LLVM Assembly Language (.ll)
- $ vim sample.ll
Compile it to LLVM Bytecode (.bc)
- $ llvm-as sample.ll
Run it on LLVM interpreter
- $ lli sample.bc

Or,

ditto
ditto
Compile it to Executable Binary File
- $ llc sample.bc
Run it!
- $ ./sample

In this post, I'll explaing about the first step "LLVM Assembly Language".

Helloworld in LLVM Assembly Language

LLVM is not a stack machine but a register machine.

(This table is from wikipedia)

Let's write helloworld application. Before that, I'll show the equivalent code in C.

int main() {
  puts("Hello, world!");
  return 0;
}

In LLVM Assembly Language, the code will be written as below.

@str = internal constant [14 x i8] c"Hello, world!\00"
declare i32 @puts(i8*)
define i32 @main()
{
  call i32 @puts( i8* getelementptr ([14 x i8]* @str, i32 0,i32 0))
  ret i32 0
}

This code suggests the following notices:

We can write an integer number directly in the assembly code, on the other hand, we cannot write a string directly.
The long name getelementptr seems to be * in C.

If I write helloworld in C like the LLVM Assembly code, it is like:

char str[14] = "Hello, world!";
int main() {
  puts((char *)str);
  return 0;
}

Fibonacci in in LLVM Assembly Language

Nanki wrote Fibonacci in LLVM Assembly Language.

@str = internal constant [4 x i8] c"%d\0A\00"

define void @main() nounwind {
init:
  br label %loop
loop:
  %i = phi i32 [0, %init], [%i.next, %loop]
  %fib = call i32 @fib(i32 %i)

  call i32 @printf( i8* getelementptr ([4 x i8]* @str, i32 0,i32 0), i32 %fib)

  %i.next = add i32 %i, 1

  %cond = icmp ult i32 %i.next, 30
  br i1 %cond, label %loop, label %exit

exit:
  ret void
}

define i32 @fib(i32 %n) nounwind {
  %cond = icmp ult i32 %n, 2
  br i1 %cond, label %c1, label %c2

c1:
  ret i32 1
c2:
  %n1 = sub i32 %n, 1
  %n2 = sub i32 %n, 2

  %fib1 = call i32 @fib(i32 %n1)
  %fib2 = call i32 @fib(i32 %n2)

  %r = add i32 %fib1, %fib2
  ret i32 %r
}

declare i32 @printf(i8*, ...) nounwind

To understand the code deeper, let me write back the code in C.

#include<stdio.h>
int fibonacci(int n);

int main() {
  int i, i_next, fib;
init:
  i = 0, i_next = 0;
loop:
  i = i_next;
  fib = fibonacci(i);
  printf("%d\n", fib);
  i_next = i + 1;
  if (i_next < 30) {
    goto loop;
  } else {
    goto exit;
  }
exit:
  return 0;
}

int fibonacci(int n) {
  int cond, n1, n2, fib1, fib2, r;
  cond = n < 2;
  if (cond) {
    goto c1;
  } else {
    goto c2;
  }
c1:
  return 1;
c2:
  n1 = n - 1;
  n2 = n - 2;
  fib1 = fibonacci(n1);
  fib2 = fibonacci(n2);
  r = fib1 + fib2;
  return r;
}

LLVM Assembly Language enables us to use the same name both for a variable and a function because of the existence of prefix
LLVM Assembly Language cannot handle many calculation at the same time like return fib(n-2) + fib(n-1).

Let's Try LLVM

2009-12-22T01:06:00.001-08:00

Mac OS X has LLVM compiler, but doesn't have LLVM Assembler. Let's start installing the trunk LLVM on your Mac.

According to http://llvm.org/docs/GettingStarted.html#checkout,

$ cd ~/src
$ svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
$ cd llvm

In the directory there is /docs directory which contains many html files. Check them.

$ ./configure --prefix=/Users/ujihisa/src/llvm/usr
$ gmake -k |& tee gnumake.out

It took long time. After the build process, I found an interesting note.

...
gmake[1]: Leaving directory `/Users/ujihisa/src/llvm/bindings'
llvm[0]: ***** Completed Debug Build
llvm[0]: ***** Note: Debug build can be 10 times slower than an
llvm[0]: ***** optimized build. Use make ENABLE_OPTIMIZED=1 to
llvm[0]: ***** make an optimized build. Alternatively you can
llvm[0]: ***** configure with --enable-optimized.

I should have set --enable-optimized.

Anyway, let the installation finish.

$ gmake install

It also took time.

Don't forget to make path to the ./usr/bin/. There are many llvm-related executable files.

Hello, world!

Let's write hello world on LLVM!

I referred this page. The sample code contains a small mistakes, so I fixed.

Write the following code on a.ll (not a.11):

@str = internal constant [13 x i8] c"hello world\0A\00"

define void @main() nounwind
{
  %temp = call i32 @printf( i8* getelementptr ([13 x i8]* @str, i32 0,i32 0))
  ret void;
}

declare i32 @printf(i8*, ...) nounwind

Assemble it:

$ llvm-as -f a.ll
$ lli a.bc
hello world

Yay!

The generated file a.bc is a binary file.

Vim

Fortunately the svn repository contains a vim script for llvm named llvm.vim. You should install it if you're a vimmer.

(To be continued...)

Today's RubySpec (Dec 21, 2009)

2009-12-21T15:34:00.001-08:00

I succeeded in fixing RubySpec to pass all String specs both in Ruby 1.8.7 and Ruby 1.9.2. yay!

$ /usr/bin/ruby ../mspec/bin/mspec ./core/string/*.rb -t ~/rubies/bin/ruby192
ruby 1.9.2dev (2009-12-21 trunk 26145) [i386-darwin9.8.0]
.............................................................................................

Finished in 0.559929 seconds

93 files, 1083 examples, 6132 expectations, 0 failures, 0 errors
$ /usr/bin/ruby ../mspec/bin/mspec ./core/string/*.rb -t ~/rubies/bin/ruby187
ruby 1.8.7 (2009-07-30 patchlevel 192) [i686-darwin9.7.0]
.............................................................................................

Finished in 0.483685 seconds

93 files, 889 examples, 5620 expectations, 0 failures, 0 errors

New knowledges for me

String#squeeze, #count and #delete receive string sequence like "a-c". In Ruby 1.8, an invalid sequence like "c-a" is just regarded as empty sequence. On the other hand, in Ruby 1.9, it raises an ArgumentError.
"\u0085" is NEL: Next Line in utf-8. On my Terminal, it looks like "\n". But actually "n" and NEL are completely different characters.
String#% Differs Between Ruby 1.8 and 1.9 (the previous blog post)

`String#%` Differs Between Ruby 1.8 and 1.9

2009-12-21T13:46:00.001-08:00

Try the following code on your ruby.

p('%-03d' % -5)

The result on ruby 1.8.* is "-05" while the result on ruby 1.9.* is "-5 ".

So, which behavior is correct? The answer is the latter.

String#% is subject to be equivalent to sprintf(3)
"-" means "left-align"
"0" means "completing the spaces with 0" when the alignment is right-align

So, "-0" is equivalent to mere "-". According to the principle, the behavior on ruby 1.8 is wrong.

(I think that the reason why ruby core developers don't change the 1.8's behavior is that the change may break existing codes.)

Efficient Software-Based Fault Isolation

2009-12-20T16:44:00.001-08:00

Paper(pdf): http://crypto.stanford.edu/cs155/papers/sfi.pdf

Discussion: http://papersincomputerscience.org/2009/12/19/efficient-software-based-fault-isolation/

Citation: Wahbe, R., Lucco, S., Anderson, T. E., and Graham, S. L. 1993. Efficient software-based fault isolation. In Proceedings of the Fourteenth ACM Symposium on Operating Systems Principles (Asheville, North Carolina, United States, December 05 – 08, 1993). SOSP ‘93. ACM, New York, NY, 203-216. (PS) (PDF)

This paper is in December 1993; 16 years ago. This paper discusses how to isolate a system failure without using any special hardwares. For example, how to impound a bug within the process is important because nobody expects that a bug of a game which is working on a system causes the whole system to crash.

This paper explains the approach with the following subsections.

Segment Matching
Address Sandboxing
Optimizations
Process Resources
Data Sharing
Implementation and Verification

Yacc, JavaCC and Racc

2009-12-20T13:33:00.001-08:00

To compare the three parser generators, here I'll show an easy sample written in them.

Target Syntax

The following codes are accepted.

"1+2"
"23423423432 + 923401"
"23432 + 2"

The compiler will calculate the single addition and shows the value.

The following codes aren't accepted.

"1+2+3"
"1-2"
"1+"

JavaCC

The following code is from Standard Compiler by Minero Aoki.

The filename is A.jj.

PARSER_BEGIN(A)
import java.io.*;

class A {
  static public void main(String[] args) {
    for (String arg : args) {
      try {
        System.out.println(evaluate(arg));
      }
      catch (ParseException ex) {
        System.err.println(ex.getMessage());
      }
    }
  }

  static public long evaluate(String src) throws ParseException {
    Reader reader = new StringReader(src);
    return new A(reader).expr();
  }
}
PARSER_END(A)

SKIP: { <[" ", "\t", "\r", "\n"]> }

TOKEN: {
  <INTEGER: (["0"-"9"])+>
}

long expr():
{
  Token x, y;
}
{
  x=<INTEGER> "+" y=<INTEGER> <EOF>
    {
      return Long.parseLong(x.image) + Long.parseLong(y.image);
    }
}

To run this, type the following commends

$ javacc A.jj
$ javac A.java
$ java A '1 +  3'
4

This build process produces the following files automatically.

A.class
A.java
AConstants.class
AConstants.java
ATokenManager.class
ATokenManager.java
ParseException.class
ParseException.java
SimpleCharStream.class
SimpleCharStream.java
Token.class
Token.java
TokenMgrError.class
TokenMgrError.java

Yacc and Lex

a.y

%token NUMBER

%%

expr : NUMBER '+' NUMBER { printf("%d", $1 + $3); }

%%
#include <stdio.h>
#include "lex.yy.c"

a.l

%{
#include "a.tab.h"
%}
%%
[0-9]+    {sscanf(yytext,"%d",&yylval); return(NUMBER);}
[ \r\n\t]   ;
.         return(yytext[0]);
%%
#ifndef yywrap
yywrap() { return 1; }
#endif

run

$ bison -d a.y && lex a.l
$ gcc a.tab.c -ly -ll
$ ./a.out

files automatically generated

a.out
a.tab.c
a.tab.h
lex.yy.c

Racc

I referred this blog entry http://d.hatena.ne.jp/morphine57/20090824/1251129740

a.racc

class A
  token NUM
rule
   expr : NUM '+' NUM { result = val[0] + val[2] }
end
---- header
require 'strscan'
---- inner
  def parse(str)
    @tokens = []
    s = StringScanner.new(str)
    until s.eos?
      case
      when s.scan(/[0-9]+/)
        @tokens << [:NUM, s[0].to_i]
      when s.skip(/[ \t\r\n]/)
      else
        @tokens << [s.getch, nil]
      end
    end
    do_parse()
  end

  def next_token
    @tokens.shift
  end

And runs

$ racc a.racc
$ ruby192 -r ./a.tab.rb -e 'p A.new.parse "1+ 2"'
3

Files automatically generated

a.tab.rb

Statistics

The lines of code which I have to write by myself:

JavaCC: 39
Yacc: 9 + 11
Racc: 26

The lines of code which are automatically generated:

JavaCC: 1446
Yacc: 3280
Racc: 117 (assuming the gem library racc is already installed)

Trying to Install JavaCC (Java Compiler Compiler)

2009-12-18T16:53:00.001-08:00

$ sudo port install javacc
--->  Computing dependencies for readline
--->  Fetching readline
--->  Attempting to fetch readline-6.0.tar.gz from http://mirrors.ibiblio.org/pub/mirrors/gnu/ftp/gnu/readline
--->  Verifying checksum(s) for readline
--->  Extracting readline
--->  Applying patches to readline
--->  Configuring readline
--->  Building readline
--->  Staging readline into destroot
--->  Deactivating readline @6.0.000_1
--->  Computing dependencies for readline
--->  Installing readline @6.0.000_2+darwin
--->  Activating readline @6.0.000_2+darwin
--->  Cleaning readline
--->  Computing dependencies for sqlite3
--->  Fetching sqlite3
--->  Attempting to fetch sqlite-3.6.21.tar.gz from http://www.sqlite.org/
--->  Verifying checksum(s) for sqlite3
--->  Extracting sqlite3
--->  Configuring sqlite3
--->  Building sqlite3
--->  Staging sqlite3 into destroot
--->  Deactivating sqlite3 @3.6.16_0
--->  Computing dependencies for sqlite3
--->  Installing sqlite3 @3.6.21_0
--->  Activating sqlite3 @3.6.21_0
--->  Cleaning sqlite3
--->  Computing dependencies for javasqlite
--->  Fetching javasqlite
--->  Attempting to fetch javasqlite-20060714.tar.gz from http://www.ch-werner.de/javasqlite/
--->  Verifying checksum(s) for javasqlite
--->  Extracting javasqlite
--->  Applying patches to javasqlite
--->  Configuring javasqlite
--->  Building javasqlite
Error: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_javasqlite/work/javasqlite-20060714" && /usr/bin/make -j1 all " returned error 2
Command output: /usr/bin/javac -nowarn  SQLite/Authorizer.java
/usr/bin/javac -nowarn  SQLite/BusyHandler.java
/usr/bin/javac -nowarn  SQLite/Callback.java
/usr/bin/javac -nowarn  SQLite/Database.java
Note: ./SQLite/TableResult.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
/usr/bin/javac -nowarn  SQLite/Shell.java
./libtool /usr/bin/gcc-4.0 -I/opt/local/include -I/opt/local/include \
            -DHAVE_SQLITE2=1 -DHAVE_SQLITE3=1 \
            -o native/mkconst native/mkconst.c /opt/local/lib/libsqlite.la /opt/local/lib/libsqlite3.la
libtool: warning: cannot infer operation mode from `/usr/bin/gcc-4.0'
libtool: you must specify a MODE
Try `libtool --help' for more information.
make: *** [native/mkconst] Error 1

Error: Status 1 encountered during processing.
/opt/local/var/macports/sources/rsync.macports.org/release/ports

readline, sqlite3, javasqlite... What was happening? I just wanted to install JavaCC...

RubyConf 2009 Was Great

2009-12-18T14:32:00.001-08:00

I had been staying in Burlingame, California to attend RubyConf 2009 and JRubyConf 2009.

I made two presentations there in English.

"Hacking parse.y"

First, I made 45 minutes presentation in English there.

Hacking parse.y (RubyConf 2009)

View more documents from ujihisa.

I talked about the parser part of MRI (Matz Ruby Implementation) with demonstrations including adding new syntaxes.

(I wouldn't like to say something negative, but I have to say that my presentation wasn't as well as I expected. I think the reasons were that I was sick at the time and I needed more safety nets of my demonstration.)

"Termtter the ultimate twitter client"

Second, I made 5 minutes short talk there.

RubyConf 2009 LT "Termtter"

View more presentations from ujihisa.

This slides were made by jugyo. He tried to make presentation, but he was so afraid of his English that he asked me to make it instead. So I did.

My roommates.

JRubyConf 2009

I also attended JRubyConf 2009 the day after RubyConf 2009. I really wanted to attend all sessions, but I couldn't because my body condition was worst at the time.

My Personal Impression

Small But Important Events

I went to Engine Yard twice. I went to eat lunch and dinner with great programmers. I talked a lot. I gave some presents to some people, and I got great things by them as well.

English

I was surprised to notice that I was starting to feel as comfortable with English as with Japanese. A few years ago, English was kind of cryptic language for me. To use English, I had to think a lot like calculating. I had to think and translate before I could understand what I was listening to.

When I lived in Canada, I didn't notice that I had stopped doing that. Maybe because my improvement was too gradual to be noticed. My one month stay in Japan made me realize this fact.

It's exciting. Even though I still can't speak English fluently or listen to English without having difficulties, but at least, now I don't feel like using English is so difficult.

Left Hand Values in Ruby

2009-12-18T13:27:00.001-08:00

Local variables, Constants, Instance variables, Class variables and global variables

a = 1
A = 1
@a = 1
@@a = 1
$a = 1

Arefs and Attributes

a[:b] = 1
# a.[](:b, 1)

a.b = 1
a::b = 1

Combine

a, A, @a, @@a, $a, a[:b], a.b =
  1, 1, 1, 1, 1, 1, 1

Opassign

a += 1
# a, b += 1, 2  raises Syntax Error!

The only one syntax which allows to write expression in left hand sides is aref. For example, a[random(10)] = 1 is accepted. Also, I can write such codes:

alias []= instance_variable_set
alias [] instance_variable_get

a = Object.new
a['@a'] = 1
p a['@a'] #=> 1
p a #=> #<Object:0x40afb0 @a=1>

Termtter Installs New Gem Libraries Automatically

2009-12-03T12:15:00.001-08:00

I wrote a new termtter plugin.

http://github.com/jugyo/termtter/blob/dbbd3d1b16933a205470125982f03a3444de3d07/lib/plugins/gem_install.rb

Once you use this plugin by plug gem_install, your termtter will install arbitrary gem libraries when they appeared on your timeline, replies or anything on your termtter. How useful it is. If you already have g, the automatic installation will be announced on your Growl.

The plugin needs more fixes and spec.

Write Implementation and Spec on the Same File

2009-12-01T00:00:00.000-08:00

-- For http://atnd.org/events/2351 day 1

Sometimes it would be preferable to write the implementation and the specific on the same single file because of the reasons including ease of maintenance and ease of distribution. The following snippet will be helpful for the purpose.

#!/usr/bin/env ruby

{implementation}

case $0
when __FILE__
  {command line interface of the implementation}
when /spec[^/]*$/
  {spec of the implementation}
end

For example, assuming a.rb is that you've written with using the template,

ruby a.rb runs {implementation} and {command line interface of the implementation}
./a.rb runs {implementation} and {command line interface of the implementation}
spec a.rb runs {implementation} and {spec of the implementation} with RSpec
require 'a' in other scripts runs only {implementation}.

Trying Stomp and Stompserver on Ruby

2009-11-28T20:45:00.001-08:00

STOMP: Streaming Text Orientated Messaging Protocol is getting important, particularly for web programmers who need to think scalability. There are Ruby implementations of STOMP, called stomp and stompserver. Here is the brief introduction of them.

INSTALL

$ gem install stomp
$ gem install stompserver

Currently it is reported that stompserver cannot be installed well on ruby 1.9. I don't know why so far.

HELLO, WORLD

Get three terminals ready.

[Terminal 1]$ mkdir tmp && cd tmp
[Terminal 1]$ stompserver

[Terminal 2]$ irb -rubygems
> require 'stomp'
> Stomp::Client.new.subscribe('/a/b') {|m| p m.body }

[Terminal 3]$ irb -rubygems
> require 'stomp'
> Stomp::Client.new.send('/a/b', 'Hello, world!')

And then check your terminal 2.

stompserver creates etc and log directories on the current directly and opens port 61613.

I'm Attending RubyConf 2009

2009-11-19T13:27:00.001-08:00

Now I'm staying at a hotel near San Francisco Airport where RubyConf 2009 will be held on Nov 19 to 21 and JRubyConf on Nov 22 as well.

As an attendee

This is my first time to attend a conference not in Japan. I had lived in Vancouver, Canada for a half year, so now I can speak and listen English better than last year, but I still feel difficulty in using English, particularly in listening. Also, I'm shy and nervous.

If you find such person, feel free to talk to him. It's me. I may have a nameplate that is written "ujihisa".

As a speaker

I'll talk about Ruby's parser. The presentation will be not for a great Ruby hacker, but for an ordinary programmer.

If you are interested in my presentation, I highly recommend you to build the Ruby trunk in advance. It's easy but takes long time.

$ mkdir ~/rubies
$ svn co (ruby trunk)
$ cd (the directory)
$ ./configure --prefix=$HOME/rubies
$ make
$ make install

As a Japanese

If you are interested in Japanese culture, feel free to ask me about it as well as about Ruby.

Contact

http://twitter.com/ujm

Playing with JRuby's Parser

2009-11-18T08:04:00.001-08:00

Apply this patch to JRuby.

diff --git a/src/org/jruby/lexer/yacc/RubyYaccLexer.java b/src/org/jruby/lexer/yacc/RubyYaccLexer.java
index dd1733e..dc591a2 100644
--- a/src/org/jruby/lexer/yacc/RubyYaccLexer.java
+++ b/src/org/jruby/lexer/yacc/RubyYaccLexer.java
@@ -1208,6 +1208,12 @@ public class RubyYaccLexer {
     private int colon(boolean spaceSeen) throws IOException {
         int c = src.read();

+        if (c == '-' && src.peek(')')) {
+          src.read();
+          lex_state = LexState.EXPR_BEG;
+          yaccValue = new Token("=>", getPosition());
+          return Tokens.tASSOC;
+        }
         if (c == ':') {
             if (isBEG() || lex_state == LexState.EXPR_CLASS || (isARG() && spaceSeen)) {
                 lex_state = LexState.EXPR_BEG;

Now you can write :-) as => in braces as Hash literals.

$ ./bin/jruby ./bin/jirb
> {1 => 2}
=> {1=>2}
> {1 :-) 2}
=> {1=>2}
> {:aaa => :bbb}
=> {:aaa=>:bbb}
> {:aaa:-):bbb}
=> {:aaa=>:bbb}

This patch is almost same as my previous patch to MRI.

diff --git a/parse.y b/parse.y
index b8d9b59..a592aed 100644
--- a/parse.y
+++ b/parse.y
@@ -7125,6 +7125,11 @@ parser_yylex(struct parser_params *parser)

       case ':':
         c = nextc();
+        if (c == '-' && peek(')') {
+            nextc();
+            lex_state = EXPR_BEG;
+            return tASSOC;
+        }
         if (c == ':') {
             if (IS_BEG() ||
                 lex_state == EXPR_CLASS || (IS_ARG() && space_seen)) {

One Month In Japan

2009-11-18T06:50:00.001-08:00

I had been in Japan for a month until yesterday. I went to my parents' home, did things related to my admission to the graduate universities, attended some conferences and workshops, and met many friends.

List of workshops I attended:

uuuu
Kansai Ruby Workshop #38
Vim Workshop #4
LiveCoing#7
Kansai Ruby Kaigi #2 at Kansai Open Forum 2009
"Working Effectively with Legacy Code" Study Group
Golang Hackathon
xUnit Workshop
Tokyo Rails Workshop #45

I made presentations at the workshops, mostly about parse.y of Ruby.

Other events:

Professor Konami and his students
Big Buddha
Hoso Hackathon
SVTour Alumni Party
Teppan lunch
ujihkn
Ramen Hackathon
Todai Hackathon
ujihkn

My Sincerely Friends

I feel too thankful for my sincerely friends to say thank you. I proud them. May luck with you!

RubyConf 2009 Schedule

2009-11-12T06:53:00.001-08:00

My talk "Hacking parse.y" will be on Thursday Nov. 18 which is the first day of RubyConf 2009.

http://rubyconf.org/pages/schedule

Accepting Both Get And Post Method in Sinatra

2009-11-04T01:00:00.001-08:00

Whe you would like to accept both GET and POST request with the same block, use the following get_or_post method.

[Before]

require 'rubygems'
require 'sinatra'

get '/' do
  'hi'
end

post '/' do
  'hi'
end

[AFTER]

require 'rubygems'
require 'sinatra'

def get_or_post(path, opts={}, &block)
  get(path, opts, &block)
  post(path, opts, &block)
end

get_or_post '/' do
  'hi'
end

4th Vim Workshop at Osaka

2009-11-01T03:09:00.001-08:00

http://atnd.org/events/1797

Today I attended a workshop about Vim. Originally I started the workshop, and kozo_ni took over from the workshop and the soul.

The number of the attendees were 10, including a person from far Yokohama.

Presentations of the workshop were broadcast on ustream. Some presentations were recorded.

38th Kansai Ruby Workshop

2009-10-30T08:11:00.001-07:00

http://jp.rubyist.net/?KansaiWorkshop38 (in Japanese)

I attended 38th Kansai Ruby Workshop today and make a presentation there. The workshop was ustreamed.

My Presentation "Hacking parse.y"

I have made the same title presentation in Vancouver.rb so far. I also have two other opportunities to make the same title presentation in Kansai Open Forum and RubyConf2009. All presentations are different because of the difference audience. Kansai Ruby Workshop is the most casual place of them, so my presentation today was be the most casual one.

The agenda was the following:

Brief self introduction
What is Ruby Core
What is Ruby's parser
Understanding colons
Adding literal
(cont.)

Note that this presentation isn't for super Ruby programmers. I think all ruby commitors already know all of what I'll talk tomorrow.

Slides, Handouts and Record

Slides are available here(slideshare).

Handouts are availabble here(pdf).

Record is available here(ustream). (in Japanese)

GRE Exam

2009-10-14T17:19:00.001-07:00

Today I took the GRE exam.

The result was too crazy to write here. I think something must be going wrong.

RubyGems on Ruby 1.9

2009-10-08T01:19:00.001-07:00

This blog post is a note of a report about the current RubyGems 1.3.5 on Ruby 1.9 for myself (including both 1.9.1 and 1.9.2) without any good reproductions or the solutions.

When I did rake192 spec on a directory which contains a Rakefile that uses jeweler, I met an enormous errors (The suffix means the Ruby version on my machine.) I've heard that anyhow gem192 uninstall update-rubygems can solve the problem, so I did it. But after that, the rake192 started to cause segfault!

The segfault is caused by a 1.9.2 marshal's bug which is already reported but not fixed yet. I gave up using 1.9.2, and then I decided to use 1.9.1 instead.

On 1.9.1 the results were exactly same except that it didn't cause a SEGV.

Downgrading RubyGems (failed!)

According to baroquebobcat,

$ gem uninstall rubygems-update
$ gem install -v x.x.x rubygems-update
$ sudo update_rubygems

does downgrade rubygems itself. So I did them with some trivial changes, but it failed again. The following is the result:

$ update_rubygems191
<internal:gem_prelude>:344:in `method_missing': undefined method `bin_path' for Gem:Module (NoMethodError)
        from /Users/ujihisa/rubies/bin/update_rubygems191:19:in `<main>'

hmm...

What's Your View?

2009-10-02T23:46:00.001-07:00

When I was a student, my supervisor often asked me the question. It confused me, which meant that I didn't understand what I was talking.

Although the relationship between understanding the contents and grasping the view is bi-directionally, it is worth trying to grasp the view of a given research first, then understand the remains. This strategy is easier, particularly for me. Never forget to look for the author's viewpoint. Ask myself: "What's the view of the research?" Now I must be able to find the goal of the research easily.

Standards

AutoKey Rocks

AutoKey

(Added on Jan 24 2012)

Compiling a Language to Whitespace

A Language

Continuous-Passing Conversion in Haskell

Implementation and description

Unlambda Interpreter in Haskell

(added at Tue Dec 20 23:57:12 PST 2011)

Read Raw POST Request Body in Compojure

Brief introduction to Compojure

Parameters

Lazy List in C

Type Inferences of Ambiguous Literals

Lossless Data Compressions

Run-length encoding

Huffman coding

BPE: Byte Pair Encoding

Optimizer Comparisons -- Skip Nops

In Haskell

In C

When the big number isn't long long

Summary

How To Compile C to IA32

URL Encodings

JavaScript (node.js)

Ruby

Python

Vim script

Haskell

Converting From/To String

ToString

Use case

Footnote

Completing Method Names in Ruby Based on Input and Output

neco-rubymf.vim

Benchmarks between Clang and GCC about Ruby 1.9.3

Building clang 3.0

Building ruby 1.9.3 with clang

Benchmark

Footnote and references

Compiling a C File, Preserving the LLVM File

In GCC

String Interpolation in Haskell

Vanrb Lightning Talk Slides: Ruby and Vim

RSense

RSense code completion

Implementation

limitations

Vim

Vim and me

RSense built-in Vim plugin

Insert-mode completions in general

neocomplcache

RSense as omni func

one more thing

String#slice of Ruby in Vim script and Haskell

in Vim script

in Haskell

A Vim Workshop in Tokyo, Japan in May

The venue

Talks

Misc

Quotes

Workshops in Amagasaki, Japan by Me

Vim Workshop#7

CoffeeScript Workshop

See also

Vim 7.3 Conceal Current Issue

Possible solutions

Delegation in Vim script

Summary

First version

How To Build MRI n Times Faster

Build the latest LLVM and Clang

A simple test

Ruby

Pitfalls

Conclusion