Stories tagged HowTo:RSS Feed

The erlang experience; tags=Programmieren, Linux, FOSS, Functional, HowTo, Erlang

This week I had to write a little tool that would collect input on different channels, via socket / netcat, via a http server, .... Calls for a parralel Design -- Maybe a good place to write something real in Erlang. While erlang itself was really nice to write -- I do like Prolog as well as the bunch of functional languages -- doing networking in erlang seems a bit special, the interfaces just aren't thin wrappers around the libc stuff.

Getting a Socket Text interface

What sounds like a easy challenge to accomplish was actually harder than expected. All I found was some way to pass binaries representing erlang code over a socket and evaluating it remotle. While it's nice that such things are as easy to do as they are it doesn't help me with my task of moving simple strings.

start() ->
    {ok, Listen} = gen_tcp:listen(51622, [binary, {packet, 0},
                                         {reuseaddr, true},
                                         {active, true}]),
    spawn(fun () -> accept_loop(Listen) end).

accept_loop(Listen) ->
    {ok, Socket} = gen_tcp:accept(Listen),
    accept_loop(Listen),
    handle(Socket)

handle(Socket) ->
    receive
        {tcp, Socket, Bin} ->
            io:format("~p~n", binary_to_list(Bin));
        {tcp_closed, Socket} ->
            Buffer
    end.

So the socket is now up and receives text as wanted. However, as we are already runnign a parralel program it would be nice to be hable to handle multiple socket connections in parralel right? For that we just need to add a spawn() at the right place. The right place is not the handle(Socket) but the accept_loop(Listen) because the process that called accept will receive all the tcp messages.

This last part was quite obvious after finding the documentation of the {active, _} properties for the socket. always here means that you'll receive all data from the socket as erlang Messages, once delivers one package and waits until it is activated again and false requires calling a method -- this would have been possible as well when forking handle(Socket).

The web server

Ok we also want a webserver. We do not want to run some webapplication inside appache or so, just do some post/get and simple pages. Erlang here provides a built-in httpd with some mod_esi that calls some function depending on the URL used. It doesn't do anything fancy like templating or DB backends or stuff, just taking strings and building the http answers.

Unfortunately there are no examples around and basically noone seems to be using this combination (apart from some hacks like mine probably). So as I needed to get some additional information into the handler function (a Pid to connect to some service), I, as a novice, just couldn't find a way. Asking on IRC the solution was rather simple: Just using erlang's process registry. For more complex stuf gproc might prove usefull here.

Result

I guess I've got a huge step farther in programming erlang now. The manpages are easily found by your search engine -- for python I have to either walk through the (well structured) online documentation or search for the right link in the search results, for erlang they're typically first. Also Joe Armstrong's Books as proven usefull. The most dificult part probably is getting around all the nice extras you can do (transfering functions over sockets et al) and find out how to do the thing you need.


-- Christoph Egger <christoph@christoph-egger.org> So, 05 Dez 2010 12:58:23 +0100

CSSH but without X; tags=Debian, Linux, FOSS, HowTo

There are many ways to run some commands simultaneously on multiple hosts like cssh or dsh. They come handy for example when you are installing software updates on a set of hosts.

dsh is a rather simple comandline tool allowing to execute a command over ssh on multiple hosts. However it doesn't allow any interactive input -- so you can't look at the potentially upgrading packages and press y to accept and you can't go through debconf promts or similar.

This is solved by cssh which opens a XTerm for every host and a input area that is broadcastet to all of them. this is working really well -- you can execute your update on all hosts and still do individual adjustments just as needed: switch focus from the broadcasted input to one of the terminal windows and anything you type just goes there.

Now cssh has a big disadvantage: it requires a running X server (and doesn't do too well with a fullscreen windowmanager). Requiring X is quite a blocker if you need to run that ssh multiplexer on a remote host, for example if the firewalling doesn't allow direct connections. Fortunately you can make tmux behave as we want -- in a simple terminal:

First you need a script spawning the ssh sessions in separate tmux panes and direct input to all of them -- here called ssh-everywhere.sh (you could also write a tmux config I guess):

#/bin/sh
# ssh-everywhere.sh
for i in $HOSTS
do
  tmux splitw "ssh $i"
  tmux select-layout tiled
done
tmux set-window-option synchronize-panes on

Now start the whole thing:

tmux new 'exec sh ssh-everywhere.sh'

And be done.

Update

If you want to type in just one pane (on one host) you can do that as well: C-b : set-window-option synchronize-panes off and moving to the right pane (C-b + Arrow keys)


-- Christoph Egger <christoph@christoph-egger.org> So, 20 Feb 2011 17:23:04 +0100

Trying GNU/Hurd; tags=Debian, HowTo, Hurd, Porting

So this is a collection of things I came about when trying to get a Debian GNU/Hurd virtual machine running with kvm. Most of it is properly documented if you manage to find that particular piece of information.

Kernel Version

Due to a bug in linux 2.6.37 and .38 hurd will only boot if you supply -no-kvm-irqchip which is not that easy if you are using libvirt. A wrapper `kvm` script in the PATH will do, as will using a 2.6.39 kernel.

sudo

sudo will hang before returning from executing some command. I'm now using screen and sudo -i which keeps you a working tty gets you root and hasn't caused mayor trouble yet

sshd

openssh-server won't come up complaining about missing PRNG – and indeed there's no /dev/{u,}random in the default install. fix is to install random-egd from ports.


-- Christoph Egger <christoph@christoph-egger.org> Fr, 06 Mai 2011 00:02:24 +0200

Marking all closed bug reports "read" in a Maildir; tags=Debian, FOSS, HowTo

#!/usr/bin/python

from btsutils.debbugs import debbugs
import mailbox
import re
import sys

mailbox =  mailbox.Maildir(sys.argv[1], factory=False)
bts = debbugs()

for key in mailbox.keys():
    message = mailbox[key]
    if not 'S' in message.get_flags():
        if message['X-Debian-PR-Message']:
            try:
                bugnr = message['X-Debian-PR-Message'].split()[1]
            except IndexError:
                continue
        else:
            test = re.search('Bug#(\d{6})', message['Subject'])
            if test:
                bugnr = test.group(0)[4:]
            else:
                continue

        try:
            bug = bts.get(bugnr)
        except AttributeError:
            print bugnr
            continue

        if bug.getStatus() == u'done':
            message.set_flags(message.get_flags() + 'S')
            mailbox[key] = message


mailbox.flush()

Run it like python cleanbugsmail.py ~/Maildir/.debian.bugs. Anyone aware of a better solution?


-- Christoph Egger <christoph@christoph-egger.org> So, 12 Jun 2011 16:53:11 +0200

RuCTFe nsaless; tags=Uni, HowTo, Security

Greetings from the FAU Security Team (FAUST), the Uni Erlangen CTF group. We were participating in the RuCTFe competition and made it to 4th place. Following is my write-up on the nsaless service, the main crypto challenge in the competition. nsaless is a nodejs webservice providing a short message service. People can post messages and their followers receive the message encrypted to their individual RSA key.

About the gameserver protocol

The gameserver created groups of 8 users on the service 7 were just following the first user (and authorized by the first user to do so) while the first user sent a tweet containing the flag. The service used 512bit RSA with 7 as public exponent. While RSA512 is certainly weak, it's strong enough to make it unfeasible to break directly.

Attacking RSA

There are some known attacks against RSA with small exponents if no proper padding is done. The most straightforward version just takes the e-th root of the cipher-text and, if the clear message was small enough, outputs that root as plain-text. As the flag was long enough to make this attack impossible, we need a somewhat improved Attack.

Håstad's Broadcast Attack

Reminder:

  • In RSA, given a plain-text A, the sender computes Aᵉ mod N to build the cipher-text B.
  • Given simultaneous congruences we can efficiently compute a x ∈ ℤ such that x satisfies all congruences using the Chinese remainder theorem.

For NSAless we actually get several such B for different N (each belonging to different users receiving the tweet because they follow the poster). This effectively means we get Aᵉ in mod N for different N. Using the Chinese remainder theorem we can now compute a x ∈ ℤ ≡ Aᵉ mod Π Nᵢ. If we use at least e different B for this we are guaranteed that x actually equals Aᵉ (in ): A needs to be smaller than N for all N used (otherwise we lose information during encryption), therefore Aᵉ needs to be smaller than Nᵉ.

Computing now the e-th root of x we get the plain-text A – the flag.

Fix

Fixing your service is easy enough, just increase e to an suitable number > 8. At the end of the contest 5 Teams had fixed this vulnerability by either using 17 or 65537.

EXPLOIT

The basic exploit is shown below. Unfortunately it needs to retrieve all tweets for all users the compute the flags which just takes too long to be feasible (at least at the end of the competition where tons of users already existed) so you would need some caching to make it actually work. Would have been a great idea to have users expire after an hour or two in the service!

#!/usr/bin/python

import httplib
import urllib
import re
import json
import pprint
import gmpy
import sys

userparse_re = re.compile('<a [^>]*>([^<]*)</a></div>\s*<div>([^<]*)</div>')
tweetparse_re = re.compile("<div id='last_tweet'>([0-9]+)</div>")
followingparse_re = re.compile('<div><a href="/[0-9]+">([0-9]+)</a></div>')

def my_parse_number(number):
    string = "%x" % number
    if len(string) != 64:
        return ""
    erg = []
    while string != '':
        erg = erg + [chr(int(string[:2], 16))]
        string = string[2:]
    return ''.join(erg)

def extended_gcd(a, b):
    x,y = 0, 1
    lastx, lasty = 1, 0

    while b:
        a, (q, b) = b, divmod(a,b)
        x, lastx = lastx-q*x, x
        y, lasty = lasty-q*y, y

    return (lastx, lasty, a)

def chinese_remainder_theorem(items):
  N = 1
  for a, n in items:
    N *= n

  result = 0
  for a, n in items:
    m = N/n
    r, s, d = extended_gcd(n, m)
    if d != 1:
      raise "Input not pairwise co-prime"
    result += a*s*m

  return result % N, N

def get_tweet(uid):
    try:
        conn = httplib.HTTPConnection("%s:48879" % sys.argv[1], timeout=60)
        conn.request("GET", "/%s" % uid)
        r1 = conn.getresponse()
        data = r1.read()
        tweet = re.findall(tweetparse_re, data)
        if len(tweet) != 1:
            return None
        followers = re.findall(followingparse_re, data)
        return tweet[0], followers
    except:
        return None

def get_users():
    conn = httplib.HTTPConnection("%s:48879" % sys.argv[1], timeout=60)
    conn.request("GET", "/users")
    r1 = conn.getresponse()
    data1 = r1.read(1024 * 1024)
    data = dict()
    for i in re.findall(userparse_re, data1)[:100]:
        userinfo = get_tweet(i[0])
        if userinfo != None:
            data[i[0]] = (json.loads(i[1].replace('&quot;', '"'))['n'], userinfo)

    return data

users = get_users()
allusers = users.keys()
masters = [ user for user in allusers if len(users[user][1][1]) > 0 ]

for test in masters:
    try:
        followers = users[test][1][1]
        data = []

        for fol in followers:
            n = int(users[fol][0])
            tweet = int(users[fol][1][0])
            data = data + [(tweet, n)]

        x, n = chinese_remainder_theorem(data)

        realnum = gmpy.mpz(x).root(7)[0].digits()
        print my_parse_number(int(realnum))
    except:
        pass

-- Christoph Egger <christoph@christoph-egger.org> Fr, 20 Dez 2013 13:59:29 +0100

[HOWTO] unsubscribe from a google group; tags=Hier, Web, Kurios, Rant, Fail, HowTo

Writing this because there seems to be no correct documentation on the relevant google websites and it turns out to be non-trivial. Our goal here is to unsubscribe from a ordinary google group.

Mails from the google group contain the quoted footer:

-- 
You received this message because you are subscribed to the Google
Groups "FOO" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to FOO+unsubscribe@googlegroups.com.
Visit this group at http://groups.google.com/group/FOO
For more options, visit https://groups.google.com/groups/opt_out.

Seems easy enough, so let's send a Mail to this FOO+unsubscribe address. Back comes a E-Mail:

From: FOO <FOO+unsubconfirm@googlegroups.com>
Subject: Unsubscribe request for FOO [{EJzZjpgFhDHd9seTdRA0}]
To: Christoph Egger <christoph@example.com>
Date: Tue, 18 Feb 2014 18:55:24 +0000 (38 minutes, 53 seconds ago)

 [Leave This Group]

Visit Go 

[Start] your own group, [visit] the help center, or [report]
abuse.

So click on the [Leave This Group] link and be done? Unfortunately not. Looking at the link you notice it's called http://groups.google.com/group/FOO/subscribe -- no token and "subscribe"? I actually want to unsubscribe! And indeed, clicking gets an Interface that offers to "Enter the email address to subscribe:" + Captcha. And whatever it does, it -- of course -- doesn't unsubscribe. (My guess is, it would actually work if you had a real google account associated with that email address and were logged in to that account but there's no way of verifying this as already the first condition is false in this case)

Now if you disable HTML completely for the email, a totally different content emerges:

Hello christoph@example.com,

We have received your request to unsubscribe from FOO. In order for us to complete the request, please reply to this email or visit the following confirmation URL:

http://groups.google.com/group/FOO/subscribe

If you have questions related to this or any other Google Group, visit the Help Center at http://groups.google.com/support/.

Thanks,

Google Groups

Still the non-functional link, however it also mentions a different solution: "please reply to this email" which was not present in the HTML mail at all. And it works.


-- Christoph Egger <christoph@christoph-egger.org> Di, 18 Feb 2014 20:37:01 +0100


valid XHTML, CSS -- Django based -- ©2008 Christoph Egger