Búsqueda/Instalación Librerías RubyGems Fácilmente

¿Cómo se puede explorar y tratar de librerías RubyGems? Hay una gran cantidad de joyas como las estrellas en el cielo. Si lo desea, estar de pie sobre los hombros de gigantes.

Una escena típica de cuando usted está buscando una librería que cumple su trabajo sería como el siguiente, por ejemplo, que busca una librería que está relacionado con DataMapper.

$ gem search dm

No consigues nada porque se le olvidó especificar -r opción.

$ gem search dm -r

Usted obtiene demasiados resultados, porque hay demasiadas joyas que incluyen "dm" en el nombre, por ejemplo "admin" algo.

De todos modos, que finalmente encontró una buena librería cuyo nombre es "dm-is-persistent_state_machine".

 $ gem install dm-is-persistent_states_machine

Este failes porque el nombre de la librería correcta es "dm-is-persistent_state_machine", mientras que ha escrito "dm-is-persistent_states_machine". Sí, eso es muy difícil de escribir correctamente cuando el nombre de la librería es muy larga. Puede ser difícil si el nombre no está en su lengua materna. (*1)

unite-gem

Una solución es usar un plugin Unite unite-gem. Después de instalar este plugin, basta con ejecutar

 :Unite gem

y el tipo nada. Verá los siguientes resultados.

y se puede reducir el número de candidatos.

a continuación, simplemente pulse <Cr>... el proceso de instalación se produjo. Usted no tiene que introducir el nombre completo de la librería.

Nota

• (*1): por ejemplo, muchas personas trataron de tipo "nokogiri", pero mal escrita como "nokogirl". Por último los autores Nokogiri realizó una joya alias "nokogirl" y todo el mundo se puso feliz. Este blog es sólo givin una versión más general de la solución.

How To Write And Show Multibyte Characters With Snap Framework

I'm playing with Snap these days. Snap is a web framework for Haskell.

The default template of application code will be like below

{-# LANGUAGE OverloadedStrings #-}
module Main where

import           Control.Applicative
import           Snap.Types
import           Snap.Util.FileServe
import           Text.Templating.Heist
import           Text.Templating.Heist.TemplateDirectory

import           Glue
import           Server


main :: IO ()
main = do
    td <- newTemplateDirectory' "templates" emptyTemplateState
    quickServer $ templateHandler td defaultReloadHandler $ \ts ->
        ifTop (writeBS "hello") <|>
        route [ ("foo", writeBS "bar")
              , ("echo/:echoparam", echoHandler)
              ] <|>
        templateServe ts <|>
        dir "static" (fileServe ".")


echoHandler :: Snap ()
echoHandler = do
    param <- getParam "echoparam"
    maybe (writeBS "must specify echo/param in URL")
          writeBS param

See the following line in main function:

        ifTop (writeBS "hello") <|>

This just shows a text written in ByteString "hello" to browser.

$ cabal install
$ {your application name} 3000
$ curl http://localhost:3000
hello

You can change the message "hello" to something and show the message on browser, but if the text is non-ascii like utf-8, you have to change other places as well. See the following patch:

diff --git snapfib.cabal snapfib.cabal
index d7a98fe..35b5c92 100644
--- snapfib.cabal
+++ snapfib.cabal
@@ -28,7 +28,8 @@ Executable snapfib
     text,
     containers,
     MonadCatchIO-transformers,
-    filepath >= 1.1 && <1.2
+    filepath >= 1.1 && <1.2,
+    utf8-string

   if impl(ghc >= 6.12.0)
     ghc-options: -threaded -Wall -fwarn-tabs -funbox-strict-fields -O2
diff --git src/Main.hs src/Main.hs
index d5b24c4..036b2db 100644
--- src/Main.hs
+++ src/Main.hs
@@ -9,13 +9,14 @@ import           Text.Templating.Heist.TemplateDirectory

 import           Glue
 import           Server
+import           Data.Text


 main :: IO ()
 main = do
     td <- newTemplateDirectory' "templates" emptyTemplateState
     quickServer $ templateHandler td defaultReloadHandler $ \ts ->
-        ifTop (writeBS "hello") <|>
+        ifTop (writeText $ pack "こんにちはこんにちは!") <|>
         route [ ("foo", writeBS "bar")
               , ("echo/:echoparam", echoHandler)
               ] <|>

The function writeBS in Snap receives ByteString and shows the message to browser as a response. This cannot handle UTF-8 strings for some reason, so instead of using writeBS with ByteString you have to use writeText with Text represented message. You can convert into Text with Data.Text.pack function.

See also:

• http://ujihisa.blogspot.com/2010/12/writing-fibonacci-web-server-in-haskell.html

Ruby-like Object-oriented Notation in Haskell

This article doesn't discuss about something practical but about something experimental.

import Prelude hiding ((.))

send :: Object -> Method -> Object
send (Fixnum value) (Method "to_s") = Str $ show value
send _ _ = undefined

(.) = send

data Object = Fixnum Int | Str String deriving Show
data Method = Method String deriving Show

main = print $ x.to_s
  where
    x = Fixnum 10
    to_s = Method "to_s"

See the line in main function definition: print $ x.to_s. This looks like Ruby.

• You can hide the definition of Haskell Prelude "." and can define your own function which name is "."; here I defined send first then made an alias of it.
• Object and sending method into an object are like just a data and a function which receives an argument of a specific type unless you think about class or prototype; or just, say, "performance."

Writing Fibonacci Web Server in Haskell

A practice: implementing a web service which receives a number by URL parameter and returns the corresponding Fibonacci sequence number.

GET
http://localhost:8000/fib/10

returns

fib(10) = 55

I used Snap Framework.

$ mkdir snapfib
$ cd snapfib
$ snap init

This generates an empty snap project files. then

$ cabal install
$ snapfib

builds the codes and installs a command snapfib in your ~/.cabal/bin/snapfib. You may run the app locally with the default port 8000; you may open the default page, hello world, on your browser.

Then implement fib sequence, a utility function, and the controller/view. This allows you to get the Fibonacci sequence number just by URL parameter.

Below is the main routine code from the repository.

{-# LANGUAGE OverloadedStrings #-}
module Main where

import           Control.Applicative
import           Snap.Types
import           Snap.Util.FileServe
import           Text.Templating.Heist
import           Text.Templating.Heist.TemplateDirectory

import           Glue
import           Server
import           Data.ByteString.Char8 (pack, unpack)

fibs :: [Integer]
fibs = 0 : 1 : zipWith (+) fibs (tail fibs)

fibInString :: Int -> String
fibInString n = "fib(" ++ show n ++ ") = " ++ show (fibs !! n)

main :: IO ()
main = do
    td <- newTemplateDirectory' "templates" emptyTemplateState
    quickServer $ templateHandler td defaultReloadHandler $ \ts ->
        ifTop (writeBS "hello world") <|>
        route [ ("foo", writeBS "bar")
              , ("fib/:n", fibHandler)
              ] <|>
        templateServe ts <|>
        dir "static" (fileServe ".")


fibHandler :: Snap ()
fibHandler = do
    param <- getParam "n"
    maybe (writeBS "must specify fib/n in URL")
          (writeBS . pack . fibInString . read . unpack) param

(writeBS . pack . fibInString . read . unpack) param is converting param :: Maybe ByteString into Int, getting a Fibonacci sequence number in String, converting into ByteString again, and passing it to writeBS function which is defined in Snap Framework.

It was not difficult or complicated to implement such a simple web service in Haskell as long as you have basic Haskell knowledges like Maybe Monad or String manipulations. The problem was, in my opinion, it took long time to build the web app. Every time you fix your code, you have to wait for the compilation before you access your web service.

Investing The Methods Of An Object On Unite

How often do you use Object#methods? That's convenient particularly on IRB.

methods on irb

This is also available on Unite.vim. Write the following code on your ~/.vimrc

let s:unite_source = {
      \ 'name': 'evalruby',
      \ 'is_volatile': 1,
      \ 'required_pattern_length': 1,
      \ 'max_candidates': 30,
      \ }

function! s:unite_source.gather_candidates(args, context)
  if a:context.input[-1:] == '.'
    let methods = split(
          \ unite#util#system(printf('ruby -e "puts %s.methods"', a:context.input[:-2])),
          \ "\n")
    call map(methods, printf("'%s' . v:val", a:context.input))
  else
    let methods = [a:context.input]
  endif
  return map(methods, '{
        \ "word": v:val,
        \ "source": "evalruby",
        \ "kind": "command",
        \ "action__command": printf("!ruby -e \"p %s\"", v:val),
        \ }')
endfunction

call unite#define_source(s:unite_source)

Execute :Unite evalruby on your Vim, and write an expression.

unite

Ruby Advent Calendar jp-en: 2010

This entry is for the event Ruby Advent Calendar jp-en: 2010. The previous post was from yhara and the next post will be from authorNari. I'm sure he will write an entry about GC.

I also had written an entry of the event last year http://ujihisa.blogspot.com/2009/11/write-implementation-and-spec-on-same.html. Time flees away without delay.

Ruby 1.9.2 in Production with Tatsuhiro Ujihisa

(This is the presentation slides set for vancouver ruby meetup)

Before the talk...

• two questions
• easy one and normal one

Question 1

An easy Ruby quiz: How to define a method without using "def" keyword? (only with pure Ruby)

def a
  :something
end
p a #=> :something

Sample answer

self.class.module_eval do
  define_method(:a) do
    :something
  end
end
p a

Or

eval 'def a; :something; end'
p a

• There's no keyword def but string containing 'def'

I think there are a lot more answers

Question 2

A normal Ruby quiz: How to assign a value to a local variable without using assignment operator "="? (only with pure Ruby)

a = :something
p a #=> :something

Sample answer 1

for a in [:something]; end

p a

• "Local variable" is very special in Ruby
• Declaring a new local variable is completely static

One more answer which is only on 1.9

Sample answer 2

/(?<a>).*/ =~ ''
eval 'a=:something'

p a

There are a match operator =~ and a string which contains =, but no assignment operator =.

Ruby 1.9.2 in Production with Tatsuhiro Ujihisa

image

This talk contains

1. What is Ruby 1.9.2?
2. Differences between 1.8..1.9
3. Differences between 1.9.1..1.9.2
4. Differences between 1.9.2..1.9.3

The summary of this talk

"Ruby 1.9.2 makes your code cleaner and easier to maintain."

Ruby versions

• Ruby 1.8.6
  • Mar 2007
• Ruby 1.8.7
  • May 2008
• Ruby 1.9.0
  • Dec 2007
• Ruby 1.9.1
  • Jan 2009
• Ruby 1.9.2
  • Aug 2010

Ruby versions

• Ruby 1.8.6
  • Mar 2007
• Ruby 1.9.0
  • Dec 2007
  • Rails 2.0
• Ruby 1.8.7
  • May 2008
• Ruby 1.9.1
  • Jan 2009
• Ruby 1.9.2
  • Aug 2010
  • Rails 3.0

Background: Ruby versions

• All of them are officialy stable version
• But Ruby 1.9 series didn't look like stable

What's new?

• Which problems did 1.9 solved?
• How can you write code easily?
• -> Examples

Problems and solutions

• Verbose hash notation
• Difficulty in process handling
• etc..

Problem 1

About syntax

Problem 1

JSON is cool. Dictionary in Erlang is cool. Hash in Ruby is...?

JavaScript or Erlang:

{a: 1, b: 2, c: 3}

Ruby:

{:a => 1, :b => 2, :c => 3}

1.9.2 new Hash syntax

This solved the problem

Ruby:

{a: 1, b: 2, c: 3}

This issue mattered particularly in..

Haml, the tool everyone is using, needs a lot of hash.

HTML:

<img src="a.jpg" width="64" height="128"></img>

Haml with ruby variables:

%img{:src => icon_path, :width => icon[:width], :height => icon[:height]}

(the below doesn't work)

%img(src=icon_path width=icon[:width] height=icon[:height])

This issue mattered particularly in..

Using 1.9.2 new hash syntax

%img{src: icon_path, width: icon[:width], height: icon[:height]}

Problem 2

• About builtin methods

Problem 2

system or `` operator lacked some important functionalities.

• Ruby is a good shell script (Rakefile!),
• Ruby has some file/process handling methods,
• But..
  • You couldn't retrieve the output or error of system
  • You could run a command asynchronously with system with "&", but couldn't kill the process directly
  • You couldn't run a command asynchronously with ``

Example

Start a Sinatra app by a Ruby script and kill the app by the script

in shell script:

#!/bin/sh
ruby sinatra_app.rb &
PID=$!
# something...
kill $PID

in ruby?

Bad solution

system 'ruby sinatra_app &'

• You cannot get the proccess ID, so you cannot kill the process

Better solution

pid = fork do
  exec 'ruby', 'sinatra_app'
end
# something..
Process.kill 'KILL', pid

• This doesn't work on NetBSD4 or Windows due to fork()

Best solution

pid = spawn 'ruby', 'sinatra_app'
# something...
Process.kill 'KILL', pid

• :)
• spawn =~ {fork + exec} or {system + &}
• portable

New system and spawn spec

system({'A' => 'b'}, 'c', in: input_io, [:out, :err] => :out)
#=> true/false

spawn(...)
#=> Fixnum

Even more..

open3 standard library

• Open3.capture2e
• powered by spawn (read the source!)

Problem 3

• Local variable shadowing (potantial bug!)

  a = :hello
  [1, 2, 3].each do |a|
    p a
  end
  p a #=> 3 (in 1.8)
  

Problem 4

• "Most libraries didn't work"
• Yes it was (particularly on 1.9.1)

now?

Active libraries work for sure!

• nokogiri
• rails

Problem 5

Installation

• install ruby, and then install rubygems, ...

rubygems is builtin!

• rake as well

Problem 6

Ruby is slow

YARV!

fib 31

• 1.8.7
  • 7.99sec
• 1.9.2
  • 0.64sec
• (jruby)
  • 3.00sec

How to make legacy code 1.9 compatible?

Changes between 1.8 and 1.9

• String isn't Enumerable
• when clause doesn't accept colon as semicolon

change between 1.9.1 and 1.9.2

• $: doesn't have the current dir.
  • require_relative is handy (but long...)

Demo: make a gem library 1.9.2 compatible

• github gem library
• github create etc..
• it's very 1.8 even though it's new
  • 0.1.0 was March 3, 2008
  • 0.4.5 was Oct 25, 2010; after 1.9.2 public release!

The summary of this talk (again)

"Ruby 1.9.2 makes your code cleaner and easier to maintain."

end

thanks!

by

• Tatsuhiro Ujihisa
• @ujm
• HootSuite Media, inc
• Ruby, Haskell, JavaScript, Vim script, etc..

hootsuite

appendix

• JRuby
  • --1.9 option
  • almost compatible with 1.9 except spawn() or etc

Conditional Operator Associativities

In Ruby:

a = true ? :a : true ? :b : :c
p a

Guess the answer! Yes, as you thought, the answer is :a.

In JavaScript:

var a = true ? 'a' : true ? 'b' : 'c';
alert(a);

Guess the answer! Yes, as you thought, the answer is a.

In Vim script:

let a = 1 ? 'a' : 1 ? 'b' : 'c'
echo a

Guess the answer! Yes, as you thought, the answer is a.

In PHP:

$a = true ? 'a' : true ? 'b' : 'c';
print_r($a);

Guess the answer! No, the answer if "b".

$a = true ? 'a' : (true ? 'b' : 'c');
print_r($a);

The code above is the equivalent code to the examples in Ruby, JS and Vim script. It's impossibly difficult to imagine why the specification is so.

Something like closure for Vim script

Vim script doesn't have lambda as a language feature, but you can fake similar thing with the following tricks.

When you want to pass a procedure with variables into a function in Vim script, how do you achieve it?

Let me explain with other language first, show a well-known function map.

Ruby(*1):

a = 10
p [1, 2, 3].map {|i| i + a }
#=> [11, 12, 13]

Python:

a = 10
print map(lambda x: x + a, [1, 2, 3])
#=> [11, 12, 13]

or more strictly

a = 10
def f(x):
    return x + a
print map(f, [1, 2, 3])
#=> [11, 12, 13]

On the other hand, Vim script doesn't have such fancy feature but have normal function and dictionary function. The former is like just a global function, scoped globally or file. The latter is like a method which has the concept of this of self in other languages.

let obj = {'a': 10}
function! obj.f(x)
  return a:x + self.a
endfunction

echo obj.f(1)
"=> 11

Note that the prefix a: means the identifier is an argument of the function.

This feature is compatible with lexical scope lambda except for the aspect of the anonymity. But it is very inconvenient that you always have to declare which variables the dictionary function will use.

The solution is to use a special variable l:.

function! Main()
  let a = 10
  let obj = copy(l:)
  function! obj.f(x)
    return a:x + self.a
  endfunction

  echo obj.f(1)
endfunction

call Main()

There are three notes for using this trick:

• l: is only available in a function.
• it can cause SEGV if you forget writing copy
• a: as well if you use not only local variables

Acknowledge

thinca and tyru taught me this trick for my problem presentation. thinca is using a kind of this trick in his product.

Footnote

The following code also works on Ruby.

a = 10
f = ->(x) { x + 1 }
p [1, 2, 3].map(&f)

Memoized recursive fibonacci in Python

A slow literal implementation of fibonacci function in Python is like the below:

def fib(n):
    return n if n < 2 else fib(n-2) + fib(n-1)

This is slow but you can make it faster with memoize technique, reducing the order.

__fib_cache = {}
def fib(n):
    if n in __fib_cache:
        return __fib_cache[n]
    else:
        __fib_cache[n] = n if n < 2 else fib(n-2) + fib(n-1)
        return __fib_cache[n]

This is fast, but obviously dirty. Fortunately Python has decorator feature that gives you much better way of writing.

def memoize(f):
    cache = {}
    def decorated_function(*args):
        if args in cache:
            return cache[args]
        else:
            cache[args] = f(*args)
            return cache[args]
    return decorated_function

@memoize
def fib(n):
    return n if n < 2 else fib(n-2) + fib(n-1)

The @memoize decorator is not only for this fib() function but also for general purpose. I referred this page. This is succinct and clean.

But I felt that the definition of memoize(), particularly decorated_function(), was too long. If the else clause consists of single return statement, you may re-write decorated_function() in 1 line.

Takeshi Abe taught me how to make the two lines into one single line.

cache[args] = f(*args)
return cache[args]

is equivalent to

cache.update({args: f(*args)}) or cache[args]

So let here is the final version of fib with memoize:

def memoize(f):
    cache = {}
    return lambda *args: cache[args] if args in cache else cache.update({args: f(*args)}) or cache[args]

@memoize
def fib(n):
    return n if n < 2 else fib(n-2) + fib(n-1)

It's fast, short, succinct, and cool.

Footnote

This approach may be against the Python golden way.

• http://twitter.com/fixedpoint_jp/status/2622882236600320
• http://twitter.com/kana1/status/2611298382643200
• http://twitter.com/kana1/status/2620769314013184
• http://twitter.com/kana1/status/2622096563769344

How to make a Unite plugin

Unite

The hottest Vim plugin these days is unite.vim.

:Unite file command opens a unite dialog that you can specify file names with inputting text.

You can search not only files but also other resources like buffers you've opened. Unite also allows users to create your own "resource" and to define corresponding actions.

Here I'll make a demonstrative unite plugin unite-colorscheme.

colorscheme

Vim, particularly GUI version of Vim implementations like MacVim, has colorscheme that allows you to change the appearance very much. You may change the colorscheme of the Vim just by :colorscheme ujihisa, but it's not trivial to find which colorschemes you already have. (*1)

unite-colorscheme.vim

https://github.com/ujihisa/unite-colorscheme

unite-colorscheme consists of the following two files.

autoload/unite/sources/colorscheme.vim:

let s:unite_source = {
      \ 'name': 'colorscheme',
      \ }

function! s:unite_source.gather_candidates(args, context)
  " [(name, dir)]
  " e.g. [('adaryn', '/Users/ujihisa/.vimbundles/ColorSamplerPack/colors'), ...]
  let colorlist = map(split(globpath(&runtimepath, 'colors/*.vim'), '\n'),
      \'[fnamemodify(v:val, ":t:r"), fnamemodify(v:val, ":h")]')

  return map(colorlist, '{
        \ "word": v:val[0],
        \ "source": "colorscheme",
        \ "kind": "colorscheme",
        \ "action__path": printf("%s/%s.vim", v:val[1], v:val[0]),
        \ "action__directory": v:val[1],
        \ }')
endfunction

function! unite#sources#colorscheme#define()
  return s:unite_source
endfunction

And autoload/unite/kinds/colorscheme.vim:

let s:kind = {
      \ 'name': 'colorscheme',
      \ 'default_action': 'execute',
      \ 'action_table': {},
      \ 'parents': [],
      \ }
let s:kind.action_table.execute = {
      \ 'is_selectable': 1,
      \ }
function! s:kind.action_table.execute.func(candidates)
  if len(a:candidates) != 1
    echo "candidates must be only one"
    return
  endif
  execute "colorscheme" a:candidates[0].word
endfunction

function! unite#kinds#colorscheme#define()
  return s:kind
endfunction

After you installed the plugin, you can search by :Unite colorscheme and set the colorscheme on the Vim just by selecting one colorscheme from the choice.

References and footnote

• (*1) neocomplcache.vim supports colorscheme completion.
• vim-ref
  • only the following two files: kinds/ref.vim and sources/ref.vim
• "How to cook a source of Unite (in Japanese)"

The Implementation of Enumerable#map of JRuby

See src/org/jruby/RubyEnumerable.java.

@JRubyMethod(name = {"map"}, frame = true, compat = CompatVersion.RUBY1_9)
public static IRubyObject map19(ThreadContext context, IRubyObject self, final Block block) {
    return collectCommon19(context, self, block, "map");
}

This looks like defining the map19 method with declaring it's the map method in Ruby, particularly in 1.9 mode. For some reason this is internally collectCommon19() not mapCommon19().

private static IRubyObject collectCommon19(ThreadContext context, IRubyObject self, final Block block, String methodName) {
    final Ruby runtime = context.getRuntime();
    if (block.isGiven()) {
        final RubyArray result = runtime.newArray();

        callEach19(runtime, context, self, new BlockCallback() {
            public IRubyObject call(ThreadContext ctx, IRubyObject[] largs, Block blk) {
                IRubyObject larg = checkArgs(runtime, largs);
                IRubyObject value = block.yield(ctx, larg);
                synchronized (result) {
                    result.append(value);
                }
                return runtime.getNil();
            }
        });
        return result;
    } else {
        return enumeratorize(runtime, self, methodName);
    }
}

This is the definition of collectCommon19. Focus on the then clause of the main if condition block. It's basically calling callEach19() with an instance of BlockCallback, redefining call(). This looks like block-passing style in Ruby.

Focus also on the line just before the callEach19(). The variable result looks like an empty Ruby array for storing values in map process. The definition of the method newArray() is the following.

public RubyArray newArray() {
    return RubyArray.newArray(this);
}

OK. How about RubyArray.newArray().

/** rb_ary_new
 *
 */
public static final RubyArray newArray(final Ruby runtime) {
    return newArray(runtime, ARRAY_DEFAULT_SIZE);
}

Ok, next. Note that the constant ARRAY_DEFAULT_SIZE is an int value.

public static final RubyArray newArray(final Ruby runtime, final int len) {
    RubyArray array = new RubyArray(runtime, len);
    RuntimeHelpers.fillNil(array.values, 0, array.values.length, runtime);
    return array;
}

Looks like [0, ..., 0] with size ARRAY_DEFAULT_SIZE. So you may wonder is it OK to define a longer array if the receiver of map has longer size than ARRAY_DEFAULT_SIZE? I also wonder.

The Problem and Solution of Haml

When I write an HTML file I always write in Haml first and generate HTML by haml command. Haml is so handy that Haml makes us free from writing closing tags like </p>. The problem in Haml is that it is very difficult to type % very often. The symbol is the second most difficult location (*1) in a keyboard.

The solution is to write the following code on you ~/.vim/ftplugin/haml.vim which let : key as % only in Haml file and when the cursor is on the top of the line.

inoremap <buffer><expr> : <SID>on_the_top() ? '%' : ':'
function! s:on_the_top()
  return col('.') == 1 || getline(line('.'))[0:col('.')] =~ '^\s*$'
endfunction

Note that I swapped : and ;. If you didn't swap them, just to swap your keyboard or the code.

• (*1) The most difficult location key is ^ (shift + 6). If you want to make a programming language which is highly difficult to code, use ^ a lot.

List of Vim plugins this computer has

Most of Vim plugins I'm using is controlled under pathogen until the day VimJolts will be released.

• ColorSamplerPack
  • A collection of colorschemes. I use zenburn in this collection or mrkn256 not in this collection.
• IndentAnything
  • Only for JavaScript indentation Support.
• blogger.vim
  • For blogging. Written by me.
• camelcasemotion
  • ,w as "go to the next capital character."
• ghc_complete
  • A neocomplcache plugin for ghc.
• gist-vim
  • To post the current buffer to gist.
• git-vim
  • To commit the current file or to view git diff.
• hootsuite
  • For HootSuite development.
• indent-haskell.vim
  • An indent file for Haskell.
• neocomplcache
  • The ultimate auto complete tool.
• quickrun
  • The ultimate quick running tool.
• repl.vim
  • To start repl in Ruby, Erlang and Haskell. Written by me.
• shadow.vim
  • The ultimate file-shadowing tool.
• stickykey.vim
  • To emulate normal keys as modifier keys.
  • I use this for CoffeeScript trick. http://vim-users.jp/2010/07/hack164/
• thinca-poslist
  • <C-o> can go back at any movements including j.
• uj-textile
  • ftdetect, ftplugin and syntax file for textile.
  • I suppose this plugin is written by someone. I wonder why this dir name contains uj prefix...
• unite.vim
  • Anything.el
• vim-altercmd
  • Automatically capitalize a Vim command in cmdwin
• vim-coffee-script
  • ftdetect, ftplugin, indent and syntax file for CoffeeScript.
• vim-markdown
  • ftdetect, ftplugin and syntax file for Markdown.
• vim-smartchr-0.1.0
  • My mappings in vimrc is full of the flavor of smartchr.
  • My favorite plugin.
• vim-smartword-0.0.2
  • An extention of w or e.
• vim-surround
  • cs([ changes the parenthes into brackets.
• vim-textobj-indent-0.0.3
  • You can select by indentation.
• vim-textobj-user-0.3.8
  • You can define your own textobj easily.
• vimclojure-2.1.2
  • A lot of supports for clojure including ftdetect, ftplugin, index and syntax.
  • This has repl support but I've never used it so far.
• vimproc
  • The :!, :r! and system() substitutor.
  • This is used by quickrun, repl.vim, vimshell and .vimrc
• vimshell
  • A shell for vim.

Syntax files

• gas
  • for Gnu Assembly language.
• haskell.vim
• rdoc.vim
  • obviously

Obsolete files

This blog post helped me finding some obsolete plugins that I should throw away.

• shim
  • An interactive Haskell shell in Vim.
  • repl.vim is a superset of this plugin.
• hints_man2
  • ?

Is Hash Comment in SQL?

A "#" is not the character to start a comment in SQL, but "--" is.

A "#" is the character to the end of the line in MySQL.

If you are a Vimmer and only use MySQL when you edit .sql files, you may want to regard the characters after #. .sql files are regarded as just SQL in Vim, but you can specify them as MySQL. Just to write the following one line on ~/.vim/ftdetect/mysql.vim.

autocmd BufRead,BufNewFile *.sql set filetype=mysql

Reading Git Log In Kindle

I bought a kindle last week and realized how awesome it was.

By the way most programmers' work begins with reading git-log before they start writing great codes. If you can read it with your kindle, having fantastic coffee, it should be amazing experience.

How to do that

The git log after you git pull with time ascending order is available with the following command (*1).

$ git log HEAD@{1}..HEAD --reverse

You can read the diffs as well, with -u option.

About coloring. Vim has diff syntax coloring system and can generate an html file with the color.

$ git log -u HEAD@{1}..HEAD --reverse | vim - -c 'TOhtml' -c 'w | qa!'

This generates Untitled.html.

$ open ./Untitled.html

And

1. Open Print dialog in your safari
2. Save PDF
3. Send the file to Kindle

That's it. Enjoy!

References

• *1 http://d.hatena.ne.jp/thinca/20100713/1278952492
• http://vim-users.jp/2010/04/hack140/

A Java Library Looks Like A JRuby Library

Problem

In the article I wrote, Introduction to JRuby - Bidirectional Communication, I described how to define a class in Java and call it in JRuby. The class written in Java is called by JRuby in a different way to a library written in pure ruby.

For example, assuming now you are making a Ruby library that has a class and a singleton method Hamburger.arrayze. If it's in Ruby you can write a single rb file hamburger.rb and write the following code.

hamburger.rb:

class Hamburger
  def self.arrayze(x)
    ['begin', x, 'end']
  end
end

and call it from another rb file:

require 'hamburger'
p Hamburger.arrayze('hello')
#=> ["begin", "hello", "end"]

You also can implement Hamburger class in pure Java.

Hamburger.java:

class Hamburger {
  static public String[] arrayze(String x) {
    String[] memo = {"begin", x, "end"};
    return memo;
  }
}

Compile it into a jar file.

$ javac Hamburger.java
$ jar cf hamburger.jar Hamburger.class

Then call the library from an rb file.

require 'hamburger.jar'
include_class Java::Hamburger

p Hamburger.arrayze('hello').to_a
#=> ["begin", "hello", "end"]

The result ran on JRuby is exactly same to the previous one, but the code is very different. You cannot delete .jar extension in require method, abbreviate the include_class declaration, or .to_a to convert Java class Object to Ruby Array of String.

Solution 1

First we should not to return a Java object but to return a Ruby array.

Git-SVN Low-Risk Practice

Assumptions

• You are supposed to work on a svn branch topic.
• You hate svn.
• You want to follow changes of svn trunk.
• You hate the merging in svn.
• You already checked out the svn repository with git-svn-clone with stdlayout option.

Wrong way

First you make a local git branch topic that'll sync with svn branch topic.

$ git checkout -b topic topic

Then you work on the branch, including git-commit a lot. Before you go home you push the changes into the main svn repository by git-svn-dcommit.

You sometimes have to see the changes in trunk.

$ git checkout master
$ git svn rebase

And merges the changes into your topic branch.

$ git checkout topic
$ git merge --no-ff master

You know that merge without --no-ff option breaks git-svn system so you did --no-ff.

Finally you finished all your work on the topic branch and tried to merge the changes into trunk.

$ git checkout master
$ git merge --no-ff topic

That causes a lot of conflicts that you have to resolve manually.

Right way

The "Wrong way" example had two failures.

• ambiguous name of topic
• merge from trunk to topic

The first one is not fatal but annoying. Every time you run a git command with specifying a branch name shows a warning message that your local git branch name is ambiguous. You should have make a local git branch in which name is different to the svn branch.

$ git checkout -b tpc topic

Next. You are not supposed to merge changes in trunk to topic branch that will be merged to trunk. That causes a lot of conflicts.

The solution is that you create another local git branch that doesn't sync svn branch directly.

$ git checkout -b t tpc

Instead of trying to merge from master to t, try to rebase from master to t. Regard the tpc branch for very temporary one.

$ git checkout master
$ git svn rebase
$ git checkout t
$ git rebase master

$ git checkout tpc
$ git svn dcommit
$ svn delete `git svn info --url`
$ git checkout t
$ git branch -d -r topic
$ git branch -D tpc
$ git svn branch topic
$ git checkout -b tpc topic

Note that this approach make the svn repository log messy. Be resigned; that's the subversion.

RSpec for Java

I think the reasons why people are using Java are in the following list.

• The product must run on JVM
• You are fixing existing code that is written in Java
• You or your co-workers love Java

Java has some testing frameworks written in Java, but it's difficult to implement something like RSpec due to the limitation of Java syntax. But actually you don't have to use a testing framework in Java but can use RSpec on JRuby, even though the product is not using JRuby.

A minimum example

Hello.java

public class Hello {
  public String world() {
    return "world!";
  }
}

hello_spec.rb

require 'java'
java_import 'Hello'

describe 'Hello' do
  it 'world' do
    Hello.new.world.should == 'world!'
  end
end

Compile Hello.java and run RSpec command.

$ javac Hello.java
$ jruby -S spec hello_spec.rb

This works.

If you didn't install RSpec yet, you can install it with the following command.

$ jruby -S gem install rspec

A bit complex example

This approach is powerful; you can use a Java bytecode which was compiled with some special parameters like -target. For example, J2ME programs needs to be compiled as Java 1.4 and needs a jar file as well. You still can spec such program.

A.java

import javax.microedition.midlet.MIDlet;

class A extends MIDlet {
  public void destroyApp(boolean unconditional) {
    notifyDestroyed();
  }
  protected void pauseApp() {
  }
  protected void startApp() {
  }
  static public String f() {
    return "sajdfksadfsd";
  }
}

a_spec.rb

require 'java'
java_import 'A'

describe 'A.f' do
  it 'is on java' do
    RUBY_DESCRIPTION.should be_include 'jruby'
  end

  it 'is a strange string' do
    A.f.should == 'sajdfksadfsd'
  end
end

run

$ javac -target 1.4 -source 1.4 -g:none -bootclasspath the-jar-file.jar A.java
$ jruby -J-cp the-jar-file.jar:. -S ~/git/jruby/bin/spec a_spec.rb

It works.

Shows Message Immediately in Haskell

You've seen such a prompt very often.

OK? [y/n]

Your cursor is just after the prompt with 1 space. If you type y the process will continue and if you type any other characters, not only n, the process will stop. This is easy to implement in Ruby.

print "OK? [y/n] "
if gets.chars.first == 'y'
  puts 'go go go'
else
  puts 'nooo'
end

It seems to be easy for Haskell beginners to implement it in Haskell like the following code.

main = do putStr "OK? [y/n] "
          s <- getLine
          case (head s) of
               'y' -> putStrLn "Nice"
               otherwise -> putStrLn "omg"

But it doesn't work appropriately due to stdout buffering. Haskell has IO library that can control buffering. Add hFlush stdout after importing the library.

import IO (hFlush, stdout)

main = do putStr "OK? [y/n] "
          hFlush stdout
          s <- getLine
          case (head s) of
               'y' -> putStrLn "Nice"
               otherwise -> putStrLn "omg"

This works.

Regular Expression Benchmarks

Most modern programming languages have regular expression engines. Here I'll compare the speed of each regular expression engines with long text matching which has backtracking. I used Ruby, Perl and Python for this examination.

A time-consuming regular expression example

/(aa)*aa/ =~ 'aa' needs backtracking. The longer the length is, the longer it would take. Let m as the length of the characters and n as the number of repeat. For example the case when n = 2 and m = 3 is

/(aa)*aa(bb)*bb(cc)*cc/ =~ 'aabbcc'

Codes

Ruby:

def time
  t = Time.now
  yield
  Time.now - t
end

def ryutorion(m, n)
  chars = (0...n).map {|i| ('a'.ord + i % 26).chr }
  regex = Regexp.new chars.map {|c| "(#{c*m})*#{c*m}" }.join
  str = chars.map {|c| c*m }.join
  time { regex =~ str }
end

Python:

import re
import time

def timeof(f):
    t = time.time()
    f()
    return time.time() - t

def ryutorion(m, n):
    chars = [ chr(ord('a') + i % 26) for i in range(n) ]
    regex = re.compile(''.join([ '(' + c*m + ')*' + c*m for c in chars ]))
    str = ''.join([ c*m for c in chars ])
    return timeof(lambda: regex.match(str))

Perl:

use warnings;
use strict;
use Time::HiRes qw(gettimeofday tv_interval);

sub gen_regex {
    my ($m, $n) = @_;

    join "", map { 
        my $s = chr(ord('a') + ($_ % 26)) x $m;
        "($s)*$s"
    } (0 .. $n - 1);
}

sub gen_input {
    my ($m, $n) = @_;

    join "", map { chr(ord('a') + ($_ % 26)) x $m } (0 .. $n - 1);
}

my $r = gen_regex 1, 1;
my $i = gen_input 1, 1;
$r = qr/$r/;
my $ts = [gettimeofday];
my $te = [gettimeofday] if($i =~ /$r/);
my $elapsed = tv_interval $ts, $te;
print "$elapsed\n";

for my $N (10, 20) {
    for my $M (10000, 20000) {
        $r = gen_regex $M, $N;
        $i = gen_input $M, $N;
        $r = qr/$r/;
        $ts = [gettimeofday];
        $te = [gettimeofday] if($i =~ /$r/);
        $elapsed = tv_interval $ts, $te;
        print "$elapsed\n";
    }
}

This perl example was written by Masahiro Kimoto.

Result

shorter test: n = {10000, 20000} = m = {10, 20}

• Ruby

  p ryutorion(10000, 10) #=> 0.000278
  p ryutorion(20000, 10) #=> 0.000698
  p ryutorion(10000, 20) #=> 0.000538
  p ryutorion(20000, 20) #=> 0.001026
  

• Python

  print ryutorion(10000, 10) #=> 0.00126099586487
  print ryutorion(20000, 10) #=> 0.00168895721436
  print ryutorion(10000, 20) #=> 0.00180792808533
  print ryutorion(20000, 20) #=> 0.00338697433472
  

• Perl

  ($M = 10000, $N = 10) #=> 0.000066
  ($M = 20000, $N = 10) #=> 0.000185
  ($M = 10000, $N = 20) #=> 0.000163
  ($M = 20000, $N = 20) #=> 0.000294
  

longer test: n = {10000, 20000} = m = {100, 200}

• Ruby (ruby 1.9.2)

  0.002721
  0.00672
  0.005441
  0.013364
  

• Python (python 2.7)

  Traceback (most recent call last):
    File "a.py", line 16, in <module>
      print ryutorion(10000, 100)
    File "a.py", line 11, in ryutorion
      regex = re.compile(''.join([ '(' + c*m + ')*' + c*m for c in chars ]))
    File "/usr/local/Cellar/python/2.7/lib/python2.7/re.py", line 190, in compile
      return _compile(pattern, flags)
    File "/usr/local/Cellar/python/2.7/lib/python2.7/re.py", line 243, in _compile
      p = sre_compile.compile(pattern, flags)
    File "/usr/local/Cellar/python/2.7/lib/python2.7/sre_compile.py", line 511, in compile
      "sorry, but this version only supports 100 named groups"
  AssertionError: sorry, but this version only supports 100 named groups
  

• Perl (perl 5.10.0)

  0.000981
  0.00269
  0.004181
  0.005484
  

Summary

Perl is the strongest. Use Perl.

Haskell liftM practice

with lists

I was trying to remember how to use Haskell. This blog post focuses only on Control.Monad.liftM function.

The type of liftM is (Monad m) => (a1 -> r) -> m a1 -> m r which evaluates the second argument with the first argument as if it's in do block.

Let me explain that with the following example. You need the list of the values which each of them are three times bigger than the original list [1, 2, 3]. The most straightforward way in Haskell is to use list comprehension.

[ x * 3 | x <- [1, 2, 3]]

List Comprehension is just a syntactic sugar of List Monad. The code is equivalent to the following code.

do x <- [1, 2, 3]
   return $ x * 3

Do notation is just a syntactic sugar of List Monad as well.

[1, 2, 3] >>= \x -> return (x * 3)

You also can make the code simpler with point free style.

[1, 2, 3] >>= return . (* 3)

Now it's time to use liftM. You can write the code with liftM in the following way.

liftM (* 3) [1, 2, 3]

This is the simplest. Note that you have to declare import Control.Monad to use liftM.

with Parsec

The code below is from Write Yourself a Scheme in 48 Hours.

parseNumber = liftM (Number . read) $ many1 digit

I rewrite it without liftM.

parseNumber = do x <- many1 digit
                 return $ Number $ read x

or

parseNumber = do x <- many1 digit
                 return $ (Number . read) x

or

parseNumber = many1 digit >>= \x -> return $ (Number . read) x

or

parseNumber = many1 digit >>= \x -> (return . Number . read) x

or

parseNumber = many1 digit >>= return $ Number . read

summary

Using liftM can make code simpler and easier to understand with its natural order of arguments.

The most difficult thing for using liftM is to write the correct spelling of liftM. I cannot remember how many times I wrote listM instead of liftM. It is very difficult and fatal issue.

Combining Java, Clojure and JRuby

JRuby can access classes implemented in pure Java code. Clojure can generate a class file. Therefore it's straightforward for JRuby to access classes implemented in Clojure.

First I define a class with a method in plain Java.

MrJava.java:

	class MrJava {
	public MrJava() {
	}
	void hi(String name) {
	System.out.println("Good afternoon " + name + ", how are you?");
	}
	}

view raw gistfile1.txt hosted with ❤ by GitHub

If you kill pid0, will the internal fork be automatically killed?

	pid0 = fork {
	pid1 = fork {
	sleep 2
	puts 'bah!'
	}
	sleep 1
	}
	Process.kill 'HUP', pid0

view raw gistfile1.txt hosted with ❤ by GitHub

Try the previous code. This doesn't show 'bah!' message. It seems to be true that internal fork will be automatically killed. But it's wrong. This code doesn't show 'bah!' message just because the internal fork didn't started yet when Process.kill was called.

Try to wait until both blocks are certainly for forked.

	pid0 = fork {
	pid1 = fork {
	sleep 2
	puts 'bah!'
	}
	sleep 1
	}
	sleep 0.1
	Process.kill 'HUP', pid0

view raw gistfile1.txt hosted with ❤ by GitHub

This code shows 'bah!' message after the main routine finished!

So, let's think about how to kill the internal process.

• The value of pid1 is not available in the outer main routine
• An external command ps is not available in Windows
• Usually the process ID pid0 and pid1 are sequential, but depending on it is dangerous

Here is the solution of it:

	pid0 = fork {
	pid1 = fork {
	sleep 2
	puts 'bah!'
	}
	at_exit { Process.kill 'HUP', pid1 }
	sleep 1
	}
	sleep 0.1
	Process.kill 'HUP', pid0

view raw gistfile1.txt hosted with ❤ by GitHub

I added a line of at_exit. This is cumbersome a little bit, but safe and easy.

With Spawn

The previous discussion was actually for getting ready. Windows doesn't have fork. We can use spawn instead of fork in most cases.

This succeeded!

One More Thing

When you have to fork a ruby process with spawn, for example in case to rewrite fork to spawn, how do you write?

fork { p Dir.pwd }

to

spawn 'ruby', '-e', 'p Dir.pwd'

is not correct. The ruby that booted the code itself is not always 'ruby'. It can be ruby19, jruby, rbx or other rubies. In this case, your the following approach.

require 'rbconfig'
spawn RbConfig::CONFIG['ruby_install_name'] + RbConfig::CONFIG['EXEEXT'], '-e', 'p Dir.pwd'

Compatible Array#flatten

Array#flatten with depth argument is useful.

[[1], [2], [[3], 4]].flatten(1)
#=> [1, 2, [3], 4]

That's been available since ruby 1.8.7, but older ruby such as ruby 1.8.6 doesn't support it.

Here is the pure-ruby implementation of Array#flatten. Use it in case.

	#!/usr/bin/ruby
	#!/opt/local/bin/ruby for 1.8.7
	#!ruby191
	#!ruby192
	if RUBY_VERSION <= '1.8.6'
	class Array
	alias orig_flatten flatten
	def flatten(depth = -1)
	if depth < 0
	orig_flatten
	elsif depth == 0
	self
	else
	inject([]) {|m, i|
	Array === i ? m + i : m << i
	}.flatten(depth - 1)
	end
	end
	end
	end
	p RUBY_VERSION
	p [1, 2, 3].flatten == [1, 2, 3]
	p [1, 2, [3]].flatten == [1, 2, 3]
	p [[1], [2], [3, 4]].flatten == [1, 2, 3, 4]
	p [[1], [2], [[3], 4]].flatten == [1, 2, 3, 4]
	p [1, 2, 3].flatten(1) == [1, 2, 3]
	p [1, 2, [3]].flatten(1) == [1, 2, 3]
	p [[1], [2], [3, 4]].flatten(1) == [1, 2, 3, 4]
	p [[1], [2], [[3], 4]].flatten(1) == [1, 2, [3], 4]
	p [[1, 2], [[[3], 4]]].flatten(1) == [1, 2, [[3], 4]]
	p [[1], [2], [[3], 4]].flatten(2) == [1, 2, 3, 4]
	p [[1, 2], [[[3], 4]]].flatten(2) == [1, 2, [3], 4]
	p [[1, 2], [[[3], 4]]].flatten(3) == [1, 2, 3, 4]
	p [[1, 2], [[[3], 4]]].flatten(-1) == [1, 2, 3, 4]
	p [[1, 2], [[[3], 4]]].flatten(0) == [[1, 2], [[[3], 4]]]

view raw gistfile1.txt hosted with ❤ by GitHub

I used it for my library spawn-for-legacy to support ruby 1.8.6.

Rails programmers don't need this code. Rails now only supports ruby newer than 1.8.7.

How To Run An External Command Asynchronously

This issue looks easy, but is actually complex.

On shell, you may run a command asynchronously just by adding & at the end of command.

$ sleep 10 &
[1] 8048

You can obtain the process ID, so you can stop the command.

$ kill 8048

How can we run an external command on Ruby?

system()

The easiest way to run an external command is system().

system 'sleep 10 &'

The return value of system() is the exit code. In this case, the system() returns true immediately after that ran, but it doesn't return process ID. To obtain the process ID, you can use $?.

system 'sleep 10 &'
p $? #=> 8050

But unfortunatelly the $? is not the processs ID of sleep, but sh -c sleep 10. There is a cussion. You cannot obtain the process ID of sleep directly.

One more problem: the notation is available only in UNIX based platform.

system() without shell

system 'sleep 10' uses shell but system 'sleep', '10' doesn't use shell. Separate the arguments. But system 'sleep', '10', '&' is not available.

Thread

To emulate '&' feature, how about using Ruby's asynchronous features? Thread is the easiest way of that.

t = Thread.start do
  system 'sleep', '10'
end
t.kill

Looks good, but it doesn't work. Thread certainly will stop, but the external command booted by the thread will still alive.

exec()

How about using another way of booting an external command? There is exec(), which roughly means the combination of system() and exit().

exec 'ls'

roughly means

system 'ls'
exit

So you think that the following code will work well.

t = Thread.start do
  exec 'sleep', '10'
end
t.kill

Unfortunatelly it doesn't work. You cannot use exec() in a child thread.

fork()

Look for another way of supporting asynchronous feature on Ruby. fork(). fork() copies the Ruby process itself.

pid = fork do
  exec 'sleep', '10'
end
Process.kill pid

It works! fork() is awesome!

But there is a bad news. Thread works on all platforms, but fork works on only some platforms. I'm sorry Windows users and NetBSD users.

spawn()

spawn = fork + exec. spawn() works on all platforms! It's perfect! But... ruby 1.8 doesn't have spawn()...

Summary

• Ruby 1.9 + UNIX: Use fork and exec, or use spawn.
• Ruby 1.8 + UNIX: Use fork and exec.
• Ruby 1.9 + Windows: Use spawn.
• Ruby 1.8 + Windows: Use other OS, other Ruby, or consider using this library

How To Avoid The Worst Case

I broke an expensive unopened wine bottle. The white wine was spilt all over my kitchen.

Yesterday I bought a wine bottle and a box of beer at a liquor store which is located far from my home. I bought them because they were on sale. To save my money, I came home on my foot, saving $1.75. Today I ate lunch and drunk a sip of beer. After the lunch, I was trying to get a snack. There was the wine bottle in front of the snack. My elbow hit the bottle, and broke it. The delicious expensive wine has gone. $14 and the heavy work has gone.

While I was wiping the floor with smelling the wine, I was thinking what was the cause and how should I do after that. This incident was happened without any other people except for me. I'm the true culprit. So, how can I avoid such tragedy in my future?

The causes are the following two. The fact I was drunk a little bit and the location of wine was not safe a little bit. Both of them are not crucial. But the incident certainly happened.

This issue can be summarized as that we should assume anything for people who are in bad condition. People often become bad because of drowsiness, alcohol, anger and depression. Assume the condition. Never put glass products on the edge, or the place people can touch easily.

All About Spawn

Spawn = Fork + Exec, but works also on platforms which don't support fork.

The usage of spawn is often said to be cryptic. Here I'll describe common cases.

• system('ls')

  pid = spawn(['ls', 'ls'])
  Process.wait(pid)
  

• system('ls .')

  pid = spawn('ls', '.')
  Process.wait(pid)
  

• system('ls . &')

  pid = spawn('ls', '.')
  

• system('ls . > a.txt')

  pid = spawn('ls', '.', :out => 'a.txt')
  Process.wait(pid)
  

• system('ls . >> a.txt')

  pid = spawn('ls', '.', :out => ['a.txt', 'a'])
  Process.wait(pid)
  

• system('ls . >& a.txt')

  pid = spawn('ls', '.', [:out, :err] => ['a.txt', 'w'])
  Process.wait(pid)
  

• IO.popen('cat a.txt') {|io| p io.read

  i, o = IO.pipe
  spawn('cat a.txt', :out => o)
  o.close
  p i.read
  

• system('make all &')

  spawn('make', 'all)
  

• Dir.chdir('a') { system 'make all &' }

  spawn('make', 'all', :chdir => 'a')
  

• Passing ENV:

  • Shell: $ AAA=1 make all &
  • Ruby: ENV['AAA'] = '1'; system('make all &')
  • Ruby with Spawn: spawn({'AAA' => '1'}, 'make', 'all')

Further information can be available in process.c in ruby trunk. Here the documentation from the file:

	/*
	* call-seq:
	* spawn([env,] command... [,options]) => pid
	* Process.spawn([env,] command... [,options]) => pid
	*
	* spawn executes specified command and return its pid.
	*
	* This method doesn't wait for end of the command.
	* The parent process should
	* use <code>Process.wait</code> to collect
	* the termination status of its child or
	* use <code>Process.detach</code> to register
	* disinterest in their status;
	* otherwise, the operating system may accumulate zombie processes.
	*
	* spawn has bunch of options to specify process attributes:
	*
	* env: hash
	* name => val : set the environment variable
	* name => nil : unset the environment variable
	* command...:
	* commandline : command line string which is passed to the standard shell
	* cmdname, arg1, ... : command name and one or more arguments (no shell)
	* [cmdname, argv0], arg1, ... : command name, argv[0] and zero or more arguments (no shell)
	* options: hash
	* clearing environment variables:
	* :unsetenv_others => true : clear environment variables except specified by env
	* :unsetenv_others => false : don't clear (default)
	* process group:
	* :pgroup => true or 0 : make a new process group
	* :pgroup => pgid : join to specified process group
	* :pgroup => nil : don't change the process group (default)
	* resource limit: resourcename is core, cpu, data, etc. See Process.setrlimit.
	* :rlimit_resourcename => limit
	* :rlimit_resourcename => [cur_limit, max_limit]
	* current directory:
	* :chdir => str
	* umask:
	* :umask => int
	* redirection:
	* key:
	* FD : single file descriptor in child process
	* [FD, FD, ...] : multiple file descriptor in child process
	* value:
	* FD : redirect to the file descriptor in parent process
	* string : redirect to file with open(string, "r" or "w")
	* [string] : redirect to file with open(string, File::RDONLY)
	* [string, open_mode] : redirect to file with open(string, open_mode, 0644)
	* [string, open_mode, perm] : redirect to file with open(string, open_mode, perm)
	* [:child, FD] : redirect to the redirected file descriptor
	* :close : close the file descriptor in child process
	* FD is one of follows
	* :in : the file descriptor 0 which is the standard input
	* :out : the file descriptor 1 which is the standard output
	* :err : the file descriptor 2 which is the standard error
	* integer : the file descriptor of specified the integer
	* io : the file descriptor specified as io.fileno
	* file descriptor inheritance: close non-redirected non-standard fds (3, 4, 5, ...) or not
	* :close_others => false : inherit fds (default for system and exec)
	* :close_others => true : don't inherit (default for spawn and IO.popen)
	*
	* If a hash is given as +env+, the environment is
	* updated by +env+ before <code>exec(2)</code> in the child process.
	* If a pair in +env+ has nil as the value, the variable is deleted.
	*
	* # set FOO as BAR and unset BAZ.
	* pid = spawn({"FOO"=>"BAR", "BAZ"=>nil}, command)
	*
	* If a hash is given as +options+,
	* it specifies
	* process group,
	* resource limit,
	* current directory,
	* umask and
	* redirects for the child process.
	* Also, it can be specified to clear environment variables.
	*
	* The <code>:unsetenv_others</code> key in +options+ specifies
	* to clear environment variables, other than specified by +env+.
	*
	* pid = spawn(command, :unsetenv_others=>true) # no environment variable
	* pid = spawn({"FOO"=>"BAR"}, command, :unsetenv_others=>true) # FOO only
	*
	* The <code>:pgroup</code> key in +options+ specifies a process group.
	* The corresponding value should be true, zero or positive integer.
	* true and zero means the process should be a process leader of a new
	* process group.
	* Other values specifies a process group to be belongs.
	*
	* pid = spawn(command, :pgroup=>true) # process leader
	* pid = spawn(command, :pgroup=>10) # belongs to the process group 10
	*
	* The <code>:rlimit_</code><em>foo</em> key specifies a resource limit.
	* <em>foo</em> should be one of resource types such as <code>core</code>.
	* The corresponding value should be an integer or an array which have one or
	* two integers: same as cur_limit and max_limit arguments for
	* Process.setrlimit.
	*
	* cur, max = Process.getrlimit(:CORE)
	* pid = spawn(command, :rlimit_core=>[0,max]) # disable core temporary.
	* pid = spawn(command, :rlimit_core=>max) # enable core dump
	* pid = spawn(command, :rlimit_core=>0) # never dump core.
	*
	* The <code>:chdir</code> key in +options+ specifies the current directory.
	*
	* pid = spawn(command, :chdir=>"/var/tmp")
	*
	* The <code>:umask</code> key in +options+ specifies the umask.
	*
	* pid = spawn(command, :umask=>077)
	*
	* The :in, :out, :err, a fixnum, an IO and an array key specifies a redirection.
	* The redirection maps a file descriptor in the child process.
	*
	* For example, stderr can be merged into stdout as follows:
	*
	* pid = spawn(command, :err=>:out)
	* pid = spawn(command, 2=>1)
	* pid = spawn(command, STDERR=>:out)
	* pid = spawn(command, STDERR=>STDOUT)
	*
	* The hash keys specifies a file descriptor
	* in the child process started by <code>spawn</code>.
	* :err, 2 and STDERR specifies the standard error stream (stderr).
	*
	* The hash values specifies a file descriptor
	* in the parent process which invokes <code>spawn</code>.
	* :out, 1 and STDOUT specifies the standard output stream (stdout).
	*
	* In the above example,
	* the standard output in the child process is not specified.
	* So it is inherited from the parent process.
	*
	* The standard input stream (stdin) can be specified by :in, 0 and STDIN.
	*
	* A filename can be specified as a hash value.
	*
	* pid = spawn(command, :in=>"/dev/null") # read mode
	* pid = spawn(command, :out=>"/dev/null") # write mode
	* pid = spawn(command, :err=>"log") # write mode
	* pid = spawn(command, 3=>"/dev/null") # read mode
	*
	* For stdout and stderr,
	* it is opened in write mode.
	* Otherwise read mode is used.
	*
	* For specifying flags and permission of file creation explicitly,
	* an array is used instead.
	*
	* pid = spawn(command, :in=>["file"]) # read mode is assumed
	* pid = spawn(command, :in=>["file", "r"])
	* pid = spawn(command, :out=>["log", "w"]) # 0644 assumed
	* pid = spawn(command, :out=>["log", "w", 0600])
	* pid = spawn(command, :out=>["log", File::WRONLY|File::EXCL|File::CREAT, 0600])
	*
	* The array specifies a filename, flags and permission.
	* The flags can be a string or an integer.
	* If the flags is omitted or nil, File::RDONLY is assumed.
	* The permission should be an integer.
	* If the permission is omitted or nil, 0644 is assumed.
	*
	* If an array of IOs and integers are specified as a hash key,
	* all the elements are redirected.
	*
	* # stdout and stderr is redirected to log file.
	* # The file "log" is opened just once.
	* pid = spawn(command, [:out, :err]=>["log", "w"])
	*
	* Another way to merge multiple file descriptors is [:child, fd].
	* \[:child, fd] means the file descriptor in the child process.
	* This is different from fd.
	* For example, :err=>:out means redirecting child stderr to parent stdout.
	* But :err=>[:child, :out] means redirecting child stderr to child stdout.
	* They differs if stdout is redirected in the child process as follows.
	*
	* # stdout and stderr is redirected to log file.
	* # The file "log" is opened just once.
	* pid = spawn(command, :out=>["log", "w"], :err=>[:child, :out])
	*
	* \[:child, :out] can be used to merge stderr into stdout in IO.popen.
	* In this case, IO.popen redirects stdout to a pipe in the child process
	* and [:child, :out] refers the redirected stdout.
	*
	* io = IO.popen(["sh", "-c", "echo out; echo err >&2", :err=>[:child, :out]])
	* p io.read #=> "out\nerr\n"
	*
	* spawn closes all non-standard unspecified descriptors by default.
	* The "standard" descriptors are 0, 1 and 2.
	* This behavior is specified by :close_others option.
	* :close_others doesn't affect the standard descriptors which are
	* closed only if :close is specified explicitly.
	*
	* pid = spawn(command, :close_others=>true) # close 3,4,5,... (default)
	* pid = spawn(command, :close_others=>false) # don't close 3,4,5,...
	*
	* :close_others is true by default for spawn and IO.popen.
	*
	* So IO.pipe and spawn can be used as IO.popen.
	*
	* # similar to r = IO.popen(command)
	* r, w = IO.pipe
	* pid = spawn(command, :out=>w) # r, w is closed in the child process.
	* w.close
	*
	* :close is specified as a hash value to close a fd individually.
	*
	* f = open(foo)
	* system(command, f=>:close) # don't inherit f.
	*
	* If a file descriptor need to be inherited,
	* io=>io can be used.
	*
	* # valgrind has --log-fd option for log destination.
	* # log_w=>log_w indicates log_w.fileno inherits to child process.
	* log_r, log_w = IO.pipe
	* pid = spawn("valgrind", "--log-fd=#{log_w.fileno}", "echo", "a", log_w=>log_w)
	* log_w.close
	* p log_r.read
	*
	* It is also possible to exchange file descriptors.
	*
	* pid = spawn(command, :out=>:err, :err=>:out)
	*
	* The hash keys specify file descriptors in the child process.
	* The hash values specifies file descriptors in the parent process.
	* So the above specifies exchanging stdout and stderr.
	* Internally, +spawn+ uses an extra file descriptor to resolve such cyclic
	* file descriptor mapping.
	*
	* See <code>Kernel.exec</code> for the standard shell.
	*/

view raw gistfile1.txt hosted with ❤ by GitHub

LLVM Workshop at Online.sg

I gave a lecture about LLVM.

The slides are available here: http://github.com/ujihisa/onsg9/blob/master/slides.md

I emphasized the following points. The optimizer of LLVM is awesome. I explained it with giving an example code.

First, I generated a very long helloworld code. To generate lavish code, I used a Brainf**k compiler. The helloworld written in Brainf**k doesn't have something like String literal, so it's natural that the code is longer than necessary, therefore the generated LLVM assembly language code would be lavish as well.

$ bfc.rb --llvm helloworld.bf > helloworld.ll

The line number of the automatically generated LLVM assembly language code was 2714. Here let me optimize the code.

$ llvms helloworld.ll -o a.bc
$ opt -O3 a.bc -o a2.bc
$ opt -O3 a2.bc -o a3.bc
$ llvm-dis a3.bc -o > a.ll

The optimized code, a.ll, is semantically equivalent to the original helloworld.ll, though the lines of code is only 25.

All attendees were surprized about the result.

Redirecting STDOUT

In writing spec in Ruby, you would often meet a situation that you have to reduce outputting messages to STDOUT.

def f
  p 123
end

describe 'f' do
  it 'outputs "123\n" to stdout' do
    ...
  end
end

How do you write it? Usually I assign a stringio object to $stdout. I guess many people use this approach.

def f
  p 123
end

# spec
require 'stringio'
results = $stdout = StringIO.new
f()
results.close_write
results.rewind
$stdout = STDOUT
p results.read #=> "123\n"

I started having an idea that IO#reopen may be a better solution of it. The code previously given looks obviously dirty.

Finally I figured out the new approach was also dirty.

def f
  p 123
end

require 'tempfile'
results = Tempfile.new('a').path
a = STDOUT.dup
STDOUT.reopen(results, 'w')
f()
STDOUT.reopen(a)
p File.read(results)

STDOUT.reopen doesn't accept an IO object, but only accepts a Filename.

capture_io of minitest

Eric Hodel and Ryan Davis mentioned about capture_io in minitest. It uses the former approache.

def capture_io
  require 'stringio'

  orig_stdout, orig_stderr         = $stdout, $stderr
  captured_stdout, captured_stderr = StringIO.new, StringIO.new
  $stdout, $stderr                 = captured_stdout, captured_stderr

  yield

  return captured_stdout.string, captured_stderr.string
ensure
  $stdout = orig_stdout
  $stderr = orig_stderr
end

Reference

http://www.ruby-forum.com/topic/187367

Eric Hodel++

Multi Process Programming With DRb

Usually Ruby's thread works well, but when you have to use thread-unsafe features like Dir.chdir, you need to use fork. How can you wait for the children processes certainly finish the given tasks? The answer is to use DRb.

I'll show an example. Prepare the following two files and run them:

server.rb

require 'drb'
x = Array.new(10, false)
def x.work(i)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end
DRb.start_service 'druby://localhost:1234', x

puts DRb.uri
DRb.thread.join

client.rb

require 'drb'
x = DRbObject.new nil, 'druby://localhost:1234'
10.times do |i|
  fork { x.work(i) }
end

def wait_until
  loop do
    yield and return
    sleep 0.1
  end
end

t = Time.now
wait_until { x.finish? }
puts "#{Time.now - t}sec."

run!

$ ruby server.rb &
$ ruby client.rb
...

The server has 10 small virtual tasks which can be finished within 1 second. The x.work(i) is the definition. Because of that, ideally the whole process finish maximally in 1sec, but practically because of reasons such as the definition os wait_until, the whole process will finish maximally about 1.1sec.

require 'drb'
n = 10

# Server
x = Array.new(n, false)
def x.work(i)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end

fork do
  DRb.start_service 'druby://localhost:1234', x
  puts DRb.uri
  DRb.thread.join
end

# Client
def wait_until
  loop do
    yield and return
    sleep 0.1
  end
end

x = DRbObject.new nil, 'druby://localhost:1234'
sleep 0.2 # FIXME

n.times do |i|
  fork { x.work(i) }
end

t = Time.now
wait_until { x.finish? }
puts "#{Time.now - t}sec."

I think you noticed that there is a dirty code in it. Yes, "sleep 0.2 # FIXME" is. I tried to find how to wait until the DRbObject certainly connected, but I couldn't find any good solution.

Without wait_until

I wrote the process at exit in the method in the server x.work in order not to use wait_until.

require 'drb'
n = 10

# Server
x = Array.new(n, false)
def x.work(i, t)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true

  if finish?
    puts "Total: #{Time.now - t}sec."
  end
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end

fork do
  DRb.start_service 'druby://localhost:1234', x
  puts DRb.uri
  DRb.thread.join
end

# Client
x = DRbObject.new nil, 'druby://localhost:1234'
sleep 0.2 # FIXME

t = Time.now
n.times do |i|
  fork do
    x.work(i, t)
  end
end

At first I used block passing style, but later I found that it's impossible to pass block through druby.

Or

Use Rinda which is a higher layer library using DRb.

How To Save Both Standard And Error Logs By Some Processes On A Same Log File

Problem

Assuming that you need to save both standard and error logs by two processes on a same file.

$ command1
mesg...
error...
mesg...
....

$ command2
mesg...
mesg...
mesg...
error..
....

How to do that? The following code looks good, but it doesn't work as you want.

$ (command1 >& log.txt) &
$ (command2 >& log.txt) &

You can find only the logs of command2.

How about trying this?

$ (command1 2>&1 >> log.txt) &
$ (command2 2>&1 >> log.txt) &

It is better, but it doesn't work as you want as well. The result is not chronically ordered because of buffering.

Solution

Do the following code:

$ (command1 2>&1 | cat >> log.txt) &
$ (command2 2>&1 | cat >> log.txt) &

It's tricky. The part of | cat >> means not to buffer the outputs.

What I Made

I need more effort to distribute it.

Introduction to Text Editors

If there were such a class titled "Introduction to Text Editors", I would be delighted to attend it.

Here is just my imagination.

• What is Text
  • Text in Natural Languages
  • Text in Programming Languages
• What is Edit
  • Open, save and buffer
  • Insert, delete and motion
  • Modes: view, insert and command
• Operation and motion
  • Operations: delete, correct, yank and user-defined operations
  • Motions: word, line and paragraph
• User interface
  • Font, size, color and location
  • Multiple windows
  • Keyboard and other input devices
• Programming aids
  • Syntax highlighting
  • Indentation
  • Execute the code
  • Hints
  • Tagjump
  • Handle related files
• Completion
  • Dictionary-based completion
  • Buffer-based completion
  • Omni completion
  • Mixed completion
• Customization
  • Any behavior is customizable
  • Key mapping
  • Plugin

The two major text editors, Vim and Emacs, have their own terms. I'd like to abstract them and give appropriate names.