LLVM Workshop at Online.sg

I gave a lecture about LLVM.

The slides are available here: http://github.com/ujihisa/onsg9/blob/master/slides.md

I emphasized the following points. The optimizer of LLVM is awesome. I explained it with giving an example code.

First, I generated a very long helloworld code. To generate lavish code, I used a Brainf**k compiler. The helloworld written in Brainf**k doesn't have something like String literal, so it's natural that the code is longer than necessary, therefore the generated LLVM assembly language code would be lavish as well.

$ bfc.rb --llvm helloworld.bf > helloworld.ll

The line number of the automatically generated LLVM assembly language code was 2714. Here let me optimize the code.

$ llvms helloworld.ll -o a.bc
$ opt -O3 a.bc -o a2.bc
$ opt -O3 a2.bc -o a3.bc
$ llvm-dis a3.bc -o > a.ll

The optimized code, a.ll, is semantically equivalent to the original helloworld.ll, though the lines of code is only 25.

All attendees were surprized about the result.

Redirecting STDOUT

In writing spec in Ruby, you would often meet a situation that you have to reduce outputting messages to STDOUT.

def f
  p 123
end

describe 'f' do
  it 'outputs "123\n" to stdout' do
    ...
  end
end

How do you write it? Usually I assign a stringio object to $stdout. I guess many people use this approach.

def f
  p 123
end

# spec
require 'stringio'
results = $stdout = StringIO.new
f()
results.close_write
results.rewind
$stdout = STDOUT
p results.read #=> "123\n"

I started having an idea that IO#reopen may be a better solution of it. The code previously given looks obviously dirty.

Finally I figured out the new approach was also dirty.

def f
  p 123
end

require 'tempfile'
results = Tempfile.new('a').path
a = STDOUT.dup
STDOUT.reopen(results, 'w')
f()
STDOUT.reopen(a)
p File.read(results)

STDOUT.reopen doesn't accept an IO object, but only accepts a Filename.

capture_io of minitest

Eric Hodel and Ryan Davis mentioned about capture_io in minitest. It uses the former approache.

def capture_io
  require 'stringio'

  orig_stdout, orig_stderr         = $stdout, $stderr
  captured_stdout, captured_stderr = StringIO.new, StringIO.new
  $stdout, $stderr                 = captured_stdout, captured_stderr

  yield

  return captured_stdout.string, captured_stderr.string
ensure
  $stdout = orig_stdout
  $stderr = orig_stderr
end

Reference

http://www.ruby-forum.com/topic/187367

Eric Hodel++

Multi Process Programming With DRb

Usually Ruby's thread works well, but when you have to use thread-unsafe features like Dir.chdir, you need to use fork. How can you wait for the children processes certainly finish the given tasks? The answer is to use DRb.

I'll show an example. Prepare the following two files and run them:

server.rb

require 'drb'
x = Array.new(10, false)
def x.work(i)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end
DRb.start_service 'druby://localhost:1234', x

puts DRb.uri
DRb.thread.join

client.rb

require 'drb'
x = DRbObject.new nil, 'druby://localhost:1234'
10.times do |i|
  fork { x.work(i) }
end

def wait_until
  loop do
    yield and return
    sleep 0.1
  end
end

t = Time.now
wait_until { x.finish? }
puts "#{Time.now - t}sec."

run!

$ ruby server.rb &
$ ruby client.rb
...

The server has 10 small virtual tasks which can be finished within 1 second. The x.work(i) is the definition. Because of that, ideally the whole process finish maximally in 1sec, but practically because of reasons such as the definition os wait_until, the whole process will finish maximally about 1.1sec.

require 'drb'
n = 10

# Server
x = Array.new(n, false)
def x.work(i)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end

fork do
  DRb.start_service 'druby://localhost:1234', x
  puts DRb.uri
  DRb.thread.join
end

# Client
def wait_until
  loop do
    yield and return
    sleep 0.1
  end
end

x = DRbObject.new nil, 'druby://localhost:1234'
sleep 0.2 # FIXME

n.times do |i|
  fork { x.work(i) }
end

t = Time.now
wait_until { x.finish? }
puts "#{Time.now - t}sec."

I think you noticed that there is a dirty code in it. Yes, "sleep 0.2 # FIXME" is. I tried to find how to wait until the DRbObject certainly connected, but I couldn't find any good solution.

Without wait_until

I wrote the process at exit in the method in the server x.work in order not to use wait_until.

require 'drb'
n = 10

# Server
x = Array.new(n, false)
def x.work(i, t)
  cost = rand
  puts "work(#{i}) costs #{cost}sec."
  sleep cost
  self[i] = true

  if finish?
    puts "Total: #{Time.now - t}sec."
  end
end

def x.finish?
  return false unless all?
  DRb.stop_service
  true
end

fork do
  DRb.start_service 'druby://localhost:1234', x
  puts DRb.uri
  DRb.thread.join
end

# Client
x = DRbObject.new nil, 'druby://localhost:1234'
sleep 0.2 # FIXME

t = Time.now
n.times do |i|
  fork do
    x.work(i, t)
  end
end

At first I used block passing style, but later I found that it's impossible to pass block through druby.

Or

Use Rinda which is a higher layer library using DRb.

How To Save Both Standard And Error Logs By Some Processes On A Same Log File

Problem

Assuming that you need to save both standard and error logs by two processes on a same file.

$ command1
mesg...
error...
mesg...
....

$ command2
mesg...
mesg...
mesg...
error..
....

How to do that? The following code looks good, but it doesn't work as you want.

$ (command1 >& log.txt) &
$ (command2 >& log.txt) &

You can find only the logs of command2.

How about trying this?

$ (command1 2>&1 >> log.txt) &
$ (command2 2>&1 >> log.txt) &

It is better, but it doesn't work as you want as well. The result is not chronically ordered because of buffering.

Solution

Do the following code:

$ (command1 2>&1 | cat >> log.txt) &
$ (command2 2>&1 | cat >> log.txt) &

It's tricky. The part of | cat >> means not to buffer the outputs.

What I Made

I need more effort to distribute it.

Introduction to Text Editors

If there were such a class titled "Introduction to Text Editors", I would be delighted to attend it.

Here is just my imagination.

• What is Text
  • Text in Natural Languages
  • Text in Programming Languages
• What is Edit
  • Open, save and buffer
  • Insert, delete and motion
  • Modes: view, insert and command
• Operation and motion
  • Operations: delete, correct, yank and user-defined operations
  • Motions: word, line and paragraph
• User interface
  • Font, size, color and location
  • Multiple windows
  • Keyboard and other input devices
• Programming aids
  • Syntax highlighting
  • Indentation
  • Execute the code
  • Hints
  • Tagjump
  • Handle related files
• Completion
  • Dictionary-based completion
  • Buffer-based completion
  • Omni completion
  • Mixed completion
• Customization
  • Any behavior is customizable
  • Key mapping
  • Plugin

The two major text editors, Vim and Emacs, have their own terms. I'd like to abstract them and give appropriate names.

How To Check If You Have The Command

For example, to assume you have the command svn and you don't have the command svn1:

which svn >&/dev/null
if [ $? == 0 ]; then
  echo You have svn
fi

which svn1 >&/dev/null
if [ $? == 1 ]; then
  echo You don\'t have svn1
fi

The Right Way of Running External Commands on Vim

Today I was struggled with one of the worst behavior of Vim script.

Try the following code on your Vim.

:!echo "#!"

Assuming you know that the command :! is to run the given command on your shell. This looks equivalent to

$ echo "#!"

on your terminal.

But Vim will show a cryptic error. That's because both # and ! are special letters for :! command.

• ! is the command previously ran. This is for the pseudo command :!! which means that running the same command again.
• # is the filename previously you edited.
• Similar in %

They may be useful for using Vim, but they make Vim script programmers crazy.