Thursday, May 30, 2013

HTTP HEAD Method in Ruby

The HTTP HEAD method is used to retrieve metadata about a resource. The precise definition in the HTTP RFC is:

The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification.

An easy way to issue a HEAD request is with telnet:

$ telnet google.com 80
HEAD / HTTP/1.0

Trying 74.125.137.138...
Connected to google.com.
Escape character is '^]'.
HEAD / HTTP/1.0

HTTP/1.0 200 OK
Date: Thu, 30 May 2013 15:34:17 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=cf14264bbb89b22a:FF=0:TM=1369928057:LM=1369928057:S=oGwwGGwnb0msliO-; expires=Sat, 30-May-2015 15:34:17 GMT; path=/; domain=.google.com
Set-Cookie: NID=67=gSJdDozeK31k5KcBf4CHxScq8j8BRe1qFd-HLKbbGEYUzXwdds12xnhbASMgn5Mqczh7XuzyVHIvi1412tZRfilVK_XppMumZEarcK_DCDsNMbd4S88yGcYBPeIyVHuY; expires=Fri, 29-Nov-2013 15:34:17 GMT; path=/; domain=.google.com; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN

Connection closed by foreign host.

It is also easy to use Rubys open-uri library to get this same metadata:

head.rb

#!/usr/bin/env ruby

require 'open-uri'

def do_head(uri)
  begin
    open(uri) do |http|
      puts "uri: #{http.base_uri}"
      puts "charset: #{http.charset}"
      puts "content-encoding: #{http.content_encoding.join(', ')}"
      puts "content-type: #{http.content_type}"
      puts "meta: "
      http.meta.keys.sort.each { |k| puts "\t#{k}: #{http.meta[k]}" }
      puts "last-modified: #{http.last_modified}"
      puts "Status: #{http.status.join(' - ')}"
    end
  rescue => e
    puts "#{e}"
  end
end

ARGV.each do |uri|
  do_head(uri)
  puts '-' * 50
end

The head.rb script allows you to specify multiple URLs, and prints out the metadata for each URL, or an error message if there is a problem accessing a URL. As an example, let's get the metadata for http://wikipedia.org and http://w3.org.

head.rb example

$ ./head.rb http://wikipedia.org http://www.w3.org
uri: http://www.wikipedia.org/
charset: utf-8
content-encoding: 
content-type: text/html
meta: 
 age: 11
 cache-control: s-maxage=3600, must-revalidate, max-age=0
 connection: close
 content-length: 47746
 content-type: text/html; charset=utf-8
 date: Thu, 30 May 2013 15:52:31 GMT
 last-modified: Thu, 23 May 2013 21:12:56 GMT
 server: Apache
 vary: Accept-Encoding
 x-cache: HIT from cp1019.eqiad.wmnet, HIT from cp1009.eqiad.wmnet
 x-cache-lookup: HIT from cp1019.eqiad.wmnet:3128, HIT from cp1009.eqiad.wmnet:80
 x-content-type-options: nosniff
last-modified: 2013-05-23 17:12:56 -0400
Status: 200 - OK

uri: http://www.w3.org
charset: utf-8
content-encoding: 
content-type: text/html
meta: 
 accept-ranges: bytes
 cache-control: max-age=600
 content-length: 32182
 content-location: Home.html
 content-type: text/html; charset=utf-8
 date: Thu, 30 May 2013 15:52:43 GMT
 etag: "7db6-4ddf0827d9380;89-3f26bd17a2f00"
 expires: Thu, 30 May 2013 16:02:43 GMT
 last-modified: Thu, 30 May 2013 14:42:38 GMT
 p3p: policyref="http://www.w3.org/2001/05/P3P/p3p.xml"
 server: Apache/2
 tcn: choice
 vary: negotiate,accept
last-modified: 2013-05-30 10:42:38 -0400
Status: 200 - OK

Wednesday, May 29, 2013

Echo Client and Server in Lua

Previously, I showed code for writing an echo client and server in Ruby. For fun, here's an implementation in Lua. The client and server classes are both provided in the Lua module echo.lua.

echo.lua


-- module setup
local modname = ...
local M = {}
_G[modname] = M
package.loaded[modname] = M

-- import section
local assert = assert
local setmetatable = setmetatable
local socket = require 'socket'
local string = string

-- no more external access beyond this point
setfenv(1, M)

-- private functions
local function add_lf(s)
    if string.sub(s, #s, #s) ~= '\n' then
        return s .. '\n'
    else
        return s
    end
end


-- echo server
EchoServer = {}

function EchoServer:serve_forever()
    local sock = socket.tcp()
    assert(sock:bind(self.host, self.port))
    sock:listen(5)
    while true do
        local client = sock:accept()
        local line = client:receive('*line')
        if line then
            client:send(add_lf(line))
        end
        client:close()
    end
end

function EchoServer:new(host, port)
    local o = {}
    o.host = host or '127.0.0.1'
    o.port = port or 8765
    setmetatable(o, self)
    self.__index = self
    return o
end

-- echo client
EchoClient = {}

function EchoClient:request(msg)
    local sock = socket.tcp()
    assert(sock:connect(self.host, self.port))
    sock:send(add_lf(msg))
    local response = sock:receive('*line')
    sock:close()
    return response
end

function EchoClient:new(host, port)
    local o = {}
    o.host = host or '127.0.0.1'
    o.port = port or 8765
    setmetatable(o, self)
    self.__index = self
    return o
end

Creating a script to run the echo server is quite simple:

echo_server_example.lua


local echo = require 'echo'
local server = echo.EchoServer:new()
server:serve_forever()

Likewise, the script for the echo client is also straight-forward:

echo_client_example.lua


local echo = require 'echo'
local client = echo.EchoClient:new()
print(client:request('hello'))
print(client:request('goodbye'))

echo_client_example.lua output


hello
goodbye

Echo Server and Client in Ruby

The echo server and client is the Hello, World! of network programming. Below is an example implementation in Ruby, where echod.rb is the server and echo.rb is the client. By default, the server listens on 127.0.0.1, port 8756, and the client connects to that address. Both programs accept two optional positional arguments, where the first is the host and the second is the port.

echod.rb


#!/usr/bin/env ruby

require 'socket'

class EchoServer
  def initialize(host='127.0.0.1', port=8765)
    @host = host
    @port = port
    @server = TCPServer.open(host, port)
  end
  def start_message
    STDOUT.puts "The server is running on #{@host}:#{@port}"
    STDOUT.puts "Press CTL-C to terminate"
  end
  def serve_forever
    start_message
    while client = @server.accept
      line = client.gets
      client.puts line
      client.close
    end
  end
end

host = '127.0.0.1'
port = 8765

if not ARGV.length.between?(0,2)
  puts 'usage: echod [HOST [PORT]]'
  Process.exit(1)
end

case ARGV.length
when 1
  host = ARGV[0]
when 2
  host = ARGV[0]
  port = ARGV[1].to_i
end

echod = EchoServer.new(host, port)
echod.serve_forever

echo.rb


#!/usr/bin/env ruby

require 'socket'

class EchoClient
  def initialize(host, port)
    @host = host
    @port = port
  end
  def request(msg)
    s = TCPSocket.open(@host, @port)
    s.puts msg
    response = s.gets.chomp
    s.close
    response
  end
end

host = '127.0.0.1'
port = 8765

if not ARGV.length.between?(0,2)
  puts 'usage: echo [host [port]]'
  Process.exit(1)
end

case ARGV.length
when 1
  host = ARGV[0]
when 2
  host = ARGV[0]
  port = ARGV[1].to_i
end

echo = EchoClient.new(host, port)
puts 'Enter an empty line to quit'
while true
  line = STDIN.gets.chomp
  if line == ''
    break
  end
  response = echo.request(line)
  STDOUT.puts response
end

Below is an example use of the two programs.

example


$ ./echod.rb &
$ ./echo.rb
Enter an empty line to quit
hello
hello
bye
bye

$

Tuesday, May 28, 2013

A Simple Binascii Module in Ruby and Lua

Whenever I'm working with binary data in Python, I inevitably find myself using the binascii module. The methods hexlify and unhexlify make it extremely easy to convert between binary data and the corresponding hexadecimal string.

I thought it would be fun to write a subset of Python's binascii module in Ruby and Lua.

binascii.rb


module Binascii
  def self.hexlify(s)
    a = []
    s.each_byte do |b|
      a << sprintf('%02X', b)
    end
    a.join
  end

  def self.unhexlify(s)
    a = s.split
    return a.pack('H*')
  end
end

binascii_example.rb


#!/usr/bin/env ruby

require 'binascii'

puts Binascii.hexlify('123')
puts Binascii.unhexlify('313233')

binascii_example.rb output


313233
123

binascii.lua


-- module setup
local modname = ...
local M = {}
_G[modname] = M
package.loaded[modname] = M

-- import section
local error = error
local string = string
local table = table
local tonumber = tonumber

-- no more external access after this point
setfenv(1, M)

---
-- Converts a string of bytes to a hexadecimal string
function hexlify(s)
    local a = {}
    for i=1,#s do
        local c = string.sub(s,i,i)
        local byte = string.byte(c)
        table.insert(a, string.format('%02X', byte))
    end
    return table.concat(a)
end

---
-- Converts a hexadecimal string to a string of bytes
function unhexlify(s)
    if #s % 2 ~= 0 then
        error('unhexlify: hexstring must contain even number of digits')
    end
    local a = {}
    for i=1,#s,2 do
        local hs = string.sub(s, i, i+1)
        local code = tonumber(hs, 16)
        if not code then
            error(string.format("unhexlify: '%s' is not avalid hex number", hs))
        end
        table.insert(a, string.char(code))
    end
    return table.concat(a)
end

binascii_example.lua


#!/usr/bin/env lua

local binascii = require 'binascii'

print(binascii.hexlify('123'))
print(binascii.unhexlify('313233'))

binascii_example.lua output


313233
123

Sunday, May 26, 2013

sort(1) in Ruby

As a simple example of using the getoptlong library (as well as an example of various array methods), here is a simple version of the UNIX sort program, implemented in Ruby.

sort.rb


#!/usr/bin/env ruby

require 'getoptlong'

class SortProgram
private
  def do_args
    parser = GetoptLong.new(
      ['-u', '--unique', GetoptLong::NO_ARGUMENT],
      ['-r', '--reverse', GetoptLong::NO_ARGUMENT]
    )

    @unique = false
    parser.each do |opt, arg|
      case opt
      when '-u'
        @unique = true
      when '-r'
        @reverse = true
      end
    end
  end

public
  def main
    do_args
    lines = []
    if ARGV.length > 0
      ARGV.each do |path|
        lines = File.readlines(path)
      end
    else
      lines = File.readlines('/dev/stdin')
    end
    lines.sort!
    if @unique
      lines.uniq!
    end
    if @reverse
      lines.reverse!
    end
    lines.each { |e| print e }
  end
end

program = SortProgram.new
program.main

Consider the sample input:

input.txt


apple
orange
banana
canteloupe
orange

We can sort the input in alphabetical order by

cat input.txt | ./sort.rb

or

./sort.rb input.txt.

The -r or --reverse flags sort in reverse-alphabetical order, and the -u or --unique flags output only unique entries.

Wednesday, May 22, 2013

Lua Metamethods

Like python's magic methods, Lua allows an object to define its behavior for certain operators. These definitions are simply functions, which are referred to as metamethods. The metamethods are placed as the values of specific keys in an object's metatable. A metatable is an ordinary Lua table. For instance, the metamethod that defines the '+' operator is installed as the value for key __add in the metatable..

In the example below, we create a Lua module that exports a class called Element that represents a value in the finite field of integers modulo 7. This is merely modulo arithmetic. Since the Element class serves as its own metatable, we can simply define the metamethods on the object itself.

finitefield.lua


-- ==============================================
-- finite field of integers modulo prime (7)
-- ==============================================

-- module setup
local modname = ...
local M = {}
_G[modname] = M
package.loaded[modname] = M

-- import section
local error = error
local getmetatable = getmetatable
local setmetatable = setmetatable
local string = string
local tostring = tostring

-- no more external access after this point
setfenv(1, M)

-- private
local MODULO = 7

-- public  
Element = {}

function Element:new(n)
    local o =  {}
    setmetatable(o, self)
    self.__index = self
    o.val = n % MODULO
    return o
end

function Element:__tostring()
    return string.format("%d", self.val)
end

function Element:__concat(s)
    return tostring(self) .. tostring(s)
end

function Element:__add(other)
    if getmetatable(other) ~= getmetatable(self) then
        error("attempt to 'add' an element with a non-element")
    end
    local val = self.val + other.val
    return Element:new(val)
end

function Element:__mul(other)
    if getmetatable(other) ~= getmetatable(self) then
        error("attempt to 'multiply' an element with a non-element")
    end
    local val = self.val * other.val
    return Element:new(val)
end

function Element:__lt(other)
    return self.val < other.val
end

function Element:__lt(other)
    return self.val < other.val
end

function Element:__eq(other)
    return self.val == other.val
end

example


#!/usr/bin/env lua

local ff = require 'finitefield'

local a = ff.Element:new(5)
local b = ff.Element:new(8)
    
local c = a + b 
print(a .. ' + ' .. b .. ' = ' .. c)

local d = a * b 
print(a .. ' * ' .. b .. ' = ' .. d)

output


5 + 1 = 6
5 * 1 = 5

Sunday, May 5, 2013

Ruby Reverse Polish Notation Calculator

In an effort to continue learning Ruby, I thought it would be fun to write a simple, reverse Polish notation (RPN) calculator.

In RPN, the operator follows the operands. An RPN expression is easy to evaluate. We scan the expression from left to right. When we see an operand, we push the operand to a stack. When we see an operator, we pop two operands from the stack, apply the operator to these pop'd values, and push the result value back on the stack. When we reach the end of the expression, the stack should have only one element, which is the result of the expression.

rpnc.rb


#!/usr/bin/env ruby

def compute(op, a, b)
  case op
  when '+'
    a + b
  when '-'
    a - b
  when '*'
    a * b
  when '/'
    a / b
  else
    raise ArgumentError.new("undefined op #{op}")
  end
end

def is_operand?(s)
  s =~ /^[-+]?[0-9]+$/
end

def is_operator?(s)
  ops = ['+', '-', '*', '/']
  ops.include?(s)
end

def evaluate(tokens)
  line = tokens.join(' ')
  operands = []
  tokens.each do |token|
    if is_operand?(token)
      operands << token.to_i
    elsif is_operator?(token)
      if operands.length < 2
        raise "bad expression '#{line}'"
      end
      operands << compute(token, operands.pop, operands.pop)
    else
      raise "bad token '#{token}'"
    end
  end
  if operands.length != 1
    raise "bad expression '#{line}'"
  end
  operands[0]
end

def parse
  line = gets.chomp
  tokens = line.split
end

def main
  tokens = parse
  puts evaluate(tokens)
end

main

example


$ ./rpnc
10 2 + 6 - 4 *
-24

Saturday, May 4, 2013

Simple Ruby Calculator

Clearly, you can directly use irb as a calculator. However, I'm learning Ruby, so I thought it would be fun to program a simple integer calculator. It's a great little example of a read-eval-print loop.

The calculator accepts input expressions of the form:

<non-negative integer> <op> <non-negative integer>

where op is one of:

+
addition
-
subtraction
*
multiplication
/
division
%
modulo
^
power

calc.rb


#!/usr/bin/env ruby

def evaluate(arg0, op, arg1)
  ans = case op
  when '+' 
    arg0 + arg1 
  when '-' 
    arg0 - arg21
  when '/' 
    arg0 / arg1
  when '*' 
    arg0 * arg1
  when '%' 
    arg0 % arg1
  when '^' 
    arg0 ** arg1
  else
    # should never happen
    "unknown op #{op}"
  end 
end

def parse_expression(line)
  regex = /\s*(\d+)\s*([+-\/*%^])\s*(\d+)\s*/
  expr = line.match(regex)
end

def eval_loop
  while true
    print 'calc> '
    $stdout.flush
    line = gets.chomp
    if line == 'exit'
      break
    end 
    expr = parse_expression(line)
    if expr
      arg0, op, arg1 = expr.captures
      puts evaluate(arg0.to_i, op, arg1.to_i)
    else
      puts 'invalid expression'
    end 
  end 
end

puts "enter 'exit' to quit the program'"
eval_loop

example


./calc.rb 
enter 'exit' to quit the program'
calc> 1 + 2
3
calc> 2 ^ 8
256
calc> 400 % 92
32
calc> exit

Ruby Pig Latin

Often, the best way to learn a new language is to start off with small, simple programs. Sure, these programs won't be very useful, but they'll be fun to write and will help build your foundation in the language. Before you know it, you'll be writing more and more complex programs in the language.

I'm relatively new to Ruby. In order to learn more about Ruby String and Array methods, I wrote a program that prompts the user for a phrase, and spits out the phrase converted to Pig Latin.

There are several variants of Pig Latin, but the standard rules are (taken from Wikipedia)

  1. For words that begin with consonant sounds, the initial consonant or consonant cluster is moved to the end of the word, and "ay" is added, as in the following examples:
    • happy --> appyhay
    • duck --> uckday
    • glove" --> oveglay
  2. For words that begin with vowel sounds or silent letter, "e;way"e; is added at the end of the word. Examples are
    • egg --> eggway
    • inbox --> inboxway
    • eight --> eightway

My program is simplistic in that it can only discern consonants, and not consonant sounds. Regardless, it was still fun to write.

piglatin.rb


!/usr/bin/env ruby

def is_consonant?(c)
  letters = ('a'..'z').to_a
  consonants = letters - ['a', 'e', 'i', 'o', 'u']
  consonants.include? c
end

def starts_with_consonant?(s)
  c = s.slice(0,1).downcase
  is_consonant?(c)
end

def get_suffix(s)
  starts_with_consonant?(s) ? 'ay' : 'way'
end

def rotate_initial_consonants!(s)
  i = 0 
  while starts_with_consonant?(s) and i < s.length
    c = s.slice!(0,1)
    s << c
    i += 1
  end 
end

def to_piglatin(word)
  piglatin = String.new(word) 
  rotate_initial_consonants!(piglatin)
  piglatin << get_suffix(word) 
end

def get_words
  input = gets.chomp
  words = input.split
end

def main
  words = get_words
  output = []
  words.each do |word|
    output << to_piglatin(word)
  end 
  puts output.join(' ')
end

main

example


$ ./piglatin.rb
Storm the castle at dusk
ormStay ethay astlecay atway uskday