Blogged by Ujihisa. Standard methods of programming and thoughts including Clojure, Vim, LLVM, Haskell, Ruby and Mathematics written by a Japanese programmer. github/ujihisa

Sunday, June 12, 2011

String\#slice of Ruby in Vim script and Haskell

A Ruby example of String#[]

str = "Oh, this is a pen."
p str[/this is a (\w+)\./, 1]

The result is "pen". Since String#[] is just an alias of String#slice, (*1)

p str.slice(/this is a (\w+)\./, 1)

The result is completely same.

in Vim script

Vim script version needs two separate processes; getting the list of first-matched string itself and containing sub matches, and then getting the specific item.

let str = "Oh, this is a pen."
echo matchlist(str, 'this is a \(\w\+\)\.')[1]

(Added at Sun Jun 12 09:26:44 PDT 2011) thinca suggested that matchstr() with \zs and \ze is very handy, particularly because of the different behavior in the case when it didn't match.

let str = "Oh, this is a pen."
echo matchstr(str, 'this is a \zs\w\+\ze\.')

in Haskell

Haskell version needs three separate processes with Text.Regex.Posix.=~; it's almost same to Vim but the default =~ behaviour is to assume the regex object has "global" option, so you have to pick which match.

import Text.Regex.Posix ((=~))
main = do
  let str = "Oh this is a pen."
  print $ head (str =~ "this is a ([a-zA-Z_]*)" :: [[String]]) !! 1

(Added at Sun Jun 12 12:54:01 PDT 2011) The following code is another example; it's safe in runtime and also this supports Vim's \zs and \ze.

import Text.Regex.PCRE ((=~))
import Data.String.Utils (replace)
import Safe (headMay, atMay)
import Data.Maybe (fromMaybe)

matchstr :: String -> String -> String
matchstr expr pat =
  let x = replace "\\zs" "(" $ replace "\\ze" ")" pat in
  fromMaybe "" $ headMay (expr =~ x :: [[String]]) >>= \y -> atMay y 1

main = print $ matchstr "this is a pen" " \\zs\\w+\\ze"

in Clojure

(Added at Thu Mar 22 17:49:40 PDT 2012)

((re-find #"this is a (\w+)\." "Oh, this is a pen.") 1)
; "pen"
  • (*1) Precisely no. Try and check the differences between them without the second argument.

Tuesday, May 31, 2011

A Vim Workshop in Tokyo, Japan in May

I organized a Vim workshop in Tokyo, Japan. The name of the workshop was ujihisa.vim. More than 40 people tried to come but 20+ people attended at this workshop due to the capacity of the room.

The venue

I went to Tokyo for this workshop.

a photo by guyon

a photo by guyon

Thanks TMI and kana1!

Talks

  1. shadow.vim with me
  2. all your vimrc is belong to us with kana1
    • picks a victim, points a lot of bad smells in the vimrc of the victim, and fix them!
    • if the length of your vimrc is less than 1000, you aren't a Vimmer yet.
    • syntax on is evil. use syntax enable.
  3. i18n with niw
    • strchars() doesn't treat multi-byte character correctly.
  4. tiny-vim with guyon
    • about Debian package "tiny-vim"
    • no built-in documentation!
  5. Vim in 5 minutes with Shougo
    • neocomplcache, unite, vimshell and vimfiler author
    • explained everything about Vim exactly within 5 minutes and extra minutes for further details
  6. vital.vim with me
    • it's amazing. use it if you are a Vim plugin author.
    • something like jQuery (from JavaScript) + Bundler (from Ruby)
    • there's an issue in vital.vim about debugging with quickrun. (*)
  7. metarw-yakiudon with sora_h
    • a metarw plugin yakiudon
    • works almost exactly same as blogger.vim but the backend Ruby library is different
    • after this presentation he talked about Ruby core development

(*) thinca, the quickrun author, fixed this issue with the following change in vital during the workshop!

Misc

Before and after the workshop some of the attendees had lunch and dinner with good Vim conversation. The topics were a lot.. vimshell, CoffeeScript, Titanium, HootSuite, Twitter, Ruby core, the ultimate programming language Zimbu, PHP vs Vim script, Emacs lisp and etc...

Quotes

  • "It's... it's not the Vim I knew..."
  • "I came to Japan from San Francisco only for attending at this workshop"
  • "Wow! HootSuite for iPad works amazingly!!!"

Monday, May 9, 2011

Workshops in Amagasaki, Japan by Me

I organized the 7th Vim workshop and a CoffeeScript workshop in Amagasaki, Japan on the last Saturday.

About 20 to 30 people attended the workshops. Even though the room had an air conditioner, the room was so hot and humid like sauna by them (and their MacBooks.)

I was supposed to make presentations with demonstrations, but unfortunately I forgot bringing my mini-DVI port for my initial version of MacBook Air. There were many MacBook Air attendees but nobody had the same version to me. I eventually gave up using my computer and just made presentation by friends' computers. Sorry for missing demonstrations.

Vim Workshop#7

There were five talks

  • Recommended Vim plugins (by me)
  • The implementation of lingr.vim (by tsukkee)
  • Bundle with Vundle (by kozo2)
  • Unite is useful, and rails.vim as well (by Sixeight)
  • All about vital.vim (by me)

I broadcasted them and recorded some of them (I just forgot recording the others.) http://ustream.tv/channel/uji

CoffeeScript Workshop

There were five talks.

  • Introduction to CoffeeScript -- syntax and usage (by me)
  • Node.js on Windows, and introduction to no.de (by cuzic)
  • iPhone app development with CoffeeScript and Titanium (by deguchi)
  • TDD with Jasmine and CoffeeScript (by http://twitter.com/nanki)
  • Details of CoffeeScript syntax (by yhara and me)
by fukui

by fukui

See also

Blog entries written by attendees. All of then are written in Japanese.

Vim

CoffeeScript

Tuesday, February 22, 2011

Vim 7.3 Conceal Current Issue

Vim 7.3 new feature Conceal has an issue that when you change your colorscheme the current conceal information will be partly removed automatically.

I found this issue with the combination of Haskell-Conceal plugin and unite-colorscheme plugin which internally uses tabpagecolorscheme plugin. Every time I moved the tab, the concealed text highlighted wrongly. After a research I figured out that it was not by changing the tab but by changing colorscheme into the same colorscheme.

First of all, the idiom to define a concealed token in a syntax file is (1) to define a syntax match {your favourite name here} "{regexp}" conceal cchar={the replacement} (2) and to link the color by highlight! link Conceal {token name}. Note that normal highlights only needs to hightlight without the bang.

The main part of the implementation of colorscheme command is defined as below which is from src/syntax.c of vim repository.

6755 /*
6756  * Load color file "name".
6757  * Return OK for success, FAIL for failure.
6758  */
6759     int
6760 load_colors(name)
6761     char_u *name;
6762 {
6763     char_u *buf;
6764     int        retval = FAIL;
6765     static int recursive = FALSE;
6766 
6767     /* When being called recursively, this is probably because setting
6768      * 'background' caused the highlighting to be reloaded.  This means it is
6769      * working, thus we should return OK. */
6770     if (recursive)
6771    return OK;
6772 
6773     recursive = TRUE;
6774     buf = alloc((unsigned)(STRLEN(name) + 12));
6775     if (buf != NULL)
6776     {
6777    sprintf((char *)buf, "colors/%s.vim", name);
6778    retval = source_runtime(buf, FALSE);
6779    vim_free(buf);
6780 #ifdef FEAT_AUTOCMD
6781    apply_autocmds(EVENT_COLORSCHEME, NULL, NULL, FALSE, curbuf);
6782 #endif
6783     }
6784     recursive = FALSE;
6785 
6786     return retval;
6787 }

Basically this function runs :source colors/{the colorscheme name}.vim and them runs autocmds already registers in Colorscheme event. This function itself looks good.

Every colorscheme files run :highlight clear in the beginning of the files. This removes all highlights except for user-defined ones. Thus it preserves syntactic information of each syntax files, but breaks conceal information given by each syntax files, because it's not treated as user-defined one but as just extended built-in one.

Possible solutions

I propose the following solutions. Each of them are independent and exclusive.

  1. Remove the procedure of removing conceal information from :highlight clear command
  2. Fix colorscheme command to recover the conceal information after changing colorscheme
  3. The behaviour is specification. Just add documentation in Vim core to persuade conceal users to add autocmd event to change conceal again after colorscheme changed.

The below is the patch for the solution (a).

diff --git src/syntax.c src/syntax.c
index 369311f..90165cf 100644
--- src/syntax.c
+++ src/syntax.c
@@ -6572,10 +6572,6 @@ static char *(highlight_init_light[]) =
  CENT("ColorColumn term=reverse ctermbg=LightRed",
       "ColorColumn term=reverse ctermbg=LightRed guibg=LightRed"),
 #endif
-#ifdef FEAT_CONCEAL
-   CENT("Conceal ctermbg=DarkGrey ctermfg=LightGrey",
-        "Conceal ctermbg=DarkGrey ctermfg=LightGrey guibg=DarkGrey guifg=LightGrey"),
-#endif
 #ifdef FEAT_AUTOCMD
  CENT("MatchParen term=reverse ctermbg=Cyan",
       "MatchParen term=reverse ctermbg=Cyan guibg=Cyan"),
@@ -6662,10 +6658,6 @@ static char *(highlight_init_dark[]) =
  CENT("MatchParen term=reverse ctermbg=DarkCyan",
       "MatchParen term=reverse ctermbg=DarkCyan guibg=DarkCyan"),
 #endif
-#ifdef FEAT_CONCEAL
-   CENT("Conceal ctermbg=DarkGrey ctermfg=LightGrey",
-        "Conceal ctermbg=DarkGrey ctermfg=LightGrey guibg=DarkGrey guifg=LightGrey"),
-#endif
 #ifdef FEAT_GUI
  "Normal gui=NONE",
 #endif

Saturday, February 5, 2011

Delegation in Vim script

In case when you define a function F1 which behaves same as a given function F2 in Vim script, you may just choose the following cases depending on the arguments of F2.

The solution is just to use call().

function! F1(...)
  return call('F2', a:000)
endfunction

But if you also want to preserve the definition of arguments? There are still solution of each cases.

Named arguments (or no arguments)

function! F1(x)
  return F2(a:x)
endfunction

Unnamed arguments

function! F1(...)
  return call('F2', a:000)
endfunction

Then you can make mixed version, using call().

function! F1(x, ...)
  return call('F2', insert(deepcopy(a:000), a:x))
endfunction

Basically this is "cons"ing a named variable into the top of the list.

Summary

call() is so handy.

Shougo Matsu, the neocomplcache and unite author, suggested me to use call() instead of the following "First version" of this blog post. That's absolutely better idea than I had. Thanks Shougo!

First version

The following sentense was legacy. I leave them just for your additional information.

Named arguments (or no arguments)

function! F1(x)
  return F2(a:x)
endfunction

Unnamed arguments

function! F1(...)
  return eval('F2(' . join(map(range(1, a:0), '"a:" . v:val'), ', ') . ')')
endfunction

This looks more difficult than it should be. Let me explain it.

a:0 is the number of the arguments of the function. If you call F1(x, y, z), a:0 is 3. You can access each arguments as a:1, a:2 and a:3.

range(1, a:0)

This becomes a list [1, 2, 3] if the number of the arguments of F1 is 3.

map(range(1, a:0), '"a:" . v:val'), ', ')

This becomes 'a:1, a:2, a:3'. Then you can make a string "F2(a:1, a:2, a:3)" dynamically, and call it by eval.

Then you can make mixed version easily.

function! F1(x, ...)
  return eval('F2(a:x, ' . join(map(range(1, a:0), '"a:" . v:val'), ', ') . ')')
endfunction

The difference is only just "F2(a:x, " which was originally "F2(".

Share

Thursday, January 27, 2011

How To Build MRI n Times Faster

Clang, a compiler front end which you can use instead of gcc, can build MRI: Matz Ruby Implementation, only if you succeeded in avoiding all pitfalls.

Build the latest LLVM and Clang

Mac OS X Snow Leopard has builtin clang command but it's too old to build ruby. First of all, get the latest clang and it's dependency, the latest LLVM.

$ svn checkout http://llvm.org/svn/llvm-project/llvm/trunk llvm && cd llvm/tools && svn checkout http://llvm.org/svn/llvm-project/cfe/trunk clang && cd ..
$ ./configure && time make

Note that it takes really long time. I recommend you not to build before you go away from keyboard for a while but to build before you go to bed.

A simple test

hello.c

#include<stdio.h>
int main() {
  puts("Hello, world!");
  return 0;
}

building on gcc and clang

$ time gcc hello.c -o hello-gcc
!!!        0.34 real         0.01 user         0.03 sys!!!

[%] /private/tmp
$ time ~/src/llvm/Debug+Asserts/bin/clang hello.c -o hello-clang
!!!        0.11 real         0.06 user         0.02 sys!!!

clang is 3 times faster than gcc to build the hello world code. The executable files work samel.

Ruby

On GCC

$ git clone https://github.com/ruby/ruby.git ruby193-gcc
$ cd ruby193-gcc
$ autoconf
$ ./configure --prefix=`pwd`/local --enable-pthread --disable-install-doc --with-out-ext=tk\* --program-suffix=193 --with-readline-dir=/usr/local --with-arch=x86_64
$ time make
...
!!!      214.93 real       191.58 user        23.57 sys!!!

On Clang

$ git clone https://github.com/ruby/ruby.git ruby193
$ cd ruby193
$ ./configure --prefix=`pwd`/local --enable-pthread --disable-install-doc --with-out-ext=tk\* --program-suffix=193 --with-readline-dir=/usr/local PATH=/Users/ujihisa/src/llvm/Debug+Asserts/bin/:$PATH CC=/Users/ujihisa/src/llvm/Debug+Asserts/bin/clang CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments --with-arch=x86_64
$ time make
!!!      932.06 real       883.44 user        31.09 sys!!!

It took 3 min 34.93 sec by GCC while it took 15 min 32.06 sec by Clang. GCC is 4.34 times faster than Clang to build MRI.

Pitfalls

If you use clang command of Mac OS X Snow Leopard, even though you'll succeed in finishing make ruby 1.9.3, but you cannot do anything by the Ruby.

$ ./local/bin/ruby -ve 1
ruby 1.9.3dev (2011-01-25 trunk 30651) [x86_64-darwin10.6.0]
!!!dyld: lazy symbol binding failed: Symbol not found: _rb_encdb_declare!!!
!!!  Referenced from: /Users/ujihisa/git/ruby193/local/lib/ruby/1.9.1/x86_64-darwin10.6.0/enc/encdb.bundle!!!
!!!  Expected in: flat namespace!!!

!!!dyld: Symbol not found: _rb_encdb_declare!!!
!!!  Referenced from: /Users/ujihisa/git/ruby193/local/lib/ruby/1.9.1/x86_64-darwin10.6.0/enc/encdb.bundle!!!
!!!  Expected in: flat namespace!!!

!!!/Users/ujihisa/git/ruby193/local/lib/ruby/1.9.1/x86_64-darwin10.6.0/enc/encdb.bundle: [BUG] Segmentation fault!!!
!!!ruby 1.9.3dev (2011-01-25 trunk 30651) [x86_64-darwin10.6.0]!!!

!!!-- Control frame information -----------------------------------------------!!!
!!!c:0002 p:-537663266 s:0004 b:0004 l:000003 d:000003 TOP   !!!
!!!c:0001 p:0000 s:0002 b:0002 l:001dd8 d:001dd8 TOP   !!!


!!!-- See Crash Report log file under ~/Library/Logs/CrashReporter or ---------!!!
!!!-- /Library/Logs/CrashReporter, for the more detail of ---------------------!!!
!!!-- C level backtrace information -------------------------------------------!!!

!!!-- Other runtime information -----------------------------------------------!!!

!!!* Loaded script: ./local/bin/ruby!!!

!!!* Loaded features:!!!

!!!    0 enumerator.so!!!

!!![NOTE]!!!
!!!You may have encountered a bug in the Ruby interpreter or extension libraries.!!!
!!!Bug reports are welcome.!!!
!!!For details: http://www.ruby-lang.org/bugreport.html!!!


!!!vimshell: signal 6(SIGABRT) "./local/bin/ruby -ve 1"!!!

Conclusion

You can build Ruby 1.9.3 by Clang. It's 0.23 times faster than building by GCC.

Wednesday, January 5, 2011

A Contributed Article to Kernel/VM Advent Calendar: VIM=VM

There are two kinds of programmers; one uses Vim and the other uses Emacs. I don't think there are positive counter opinions about that Emacs is not an editor but an environment. On the other hand, there are some opinions that Vim is an editor or not, even though everyone agrees with the fact that vi is an editor. Some people, including the author of this article, claim that Vim is a virtual machine. Here I give you instructions of a Vim plugin shadow.vim instead of explaining why Vim is VM directly.

shadow.vim

The Vim plugin shadow.vim supports wrapping a virtual file with a program automatically with thin configuration.

Here it's a quote from a programmer:

"Nobody knows the Java code you committed is originally written in Scheme."

Usage

Assuming the product is a.pl, create a.pl.shd first.

a.pl (in Vim):

## ruby -e 'puts $<.read.gsub(/$/, ";")'
$a = 1
print($a)

Open a.pl in Vim. The Vim actually shows the contents of a.pl.shd. When you save the file, the command in the first line without ## runs, then the actual a.pl will be the result.

a.pl (actually):

$a = 1;
print($a);

Install

Unarchive the zip file into a directory that is under &rtp of your Vim, which stands for run time path, including ~/.vim dir.

Use Case

Here there are three examples, but you can use more general purposes.

  • Commit JavaScript files which was written in CoffeeScript

    • before

          ## coffee -csb
          f = (x) -> x + 1
          print f 10
      
          # vim: set ft=coffee :
      
    • after

          var f;
          f = function(x) {
            return x + 1;
          };
          print(f(10));
      
  • Use cpp command before committing Java files.
  • Markdown, Haml or something else to HTML

More examples

You can write code in C and save the file as GNU Assembly Language on your Linux automatically, so you can avoid portability.

References

Followers