2007-03-27

The Myth of Sysifus (sic)

Here's a cute little snippet that makes my life a lot easier when interacting with the outside world in Ruby.

def sysify script
script.each_line do |l|
system l.strip
end
end


This simple little function allows me to do things like this:

def do_shell_stuff
begin
sysify <<-SETUP
# THIS IS A SHELL SCRIPT
cat /tmp/some.junk | some | util | chain > /tmp/desired.file
run_other_util /tmp/desired.file
do_lots_of_other_shell_stuff
SETUP

# THIS IS NORMAL RUBY CODE
lines = []
open("/tmp/desired.file", "r") do |f|
f.readlines.each { |l| lines << l.strip }
end

ensure
sysify <<-CLEANUP
# THIS IS A SHELL SCRIPT AGAIN
rm /tmp/desired.file
CLEANUP
end
end


I can see this being particularly useful for use in Rake actions involving software builds where you may have to in one action untar a file, run ./configure, execute a make and then a make install. Doing it with repeated system calls breaks DRY seriously where using sysify puts the calls to system in one place. The only downside I can see is that error handling could start to get hairy: specifically what happens if you expect an error return that's actually a warning from an embedded command?

I'll be thinking on that last one next.

2007-03-26

Cooking with Gas

Last post I introduced some code that allowed external commands to be run conveniently in any user context desired. Specifically, in the case of that code, I allowed it to be run with a given effective group and user ID as well as a passed-in path.

This post extends on the previous by cleaning up the code and by rendering the provided class more Ruby-like.

The problem with the code provided is that it doesn't do some things well, and it does other things well, but not in the Ruby way. The new code is attached in its entirety to the bottom of this post and we'll be referring to it in pieces here.

So, first problem: this code simply won't work properly if you're not the root user. I'd rather have this reported up-front when making an execution context rather than deeper in the code when trying to use it. I'd also like the error to be specific to the problem rather than something useless like "process failed". So the new initialize function is this:

  def initialize username, path = nil
raise SecurityError , "must be used from root privileges", caller if Process.euid != 0
require 'etc'
@path = path
@username = username
@userinfo = Etc.getpwnam(username)
end


Note the other change here. I no longer have a specific function to extract just the uid and gid of the given user. I instead take back all the stuff from getpwnam() and store it. This was done mostly to just get rid of an unnecessary function given that the user info doesn't take much space and it's just as easy to set, say, a gid to @userinfo.gid as it is to set it to @gid.

I'm all about the reduced typing.

Another (cosmetic) change is that the store and restore methods have been made private. Further the actual code that swapped the context for execution has been moved to a method and made private as well. This is simple reorganisation stuff, however. Here's the biggest change:

  def execute
store
switch_context
begin
yield
ensure
restore
end
end


This code is far more the Ruby Way than my previous efforts (although I will undoubtedly find other ways to improve on it as time goes on). Instead of taking a string, swapping context and then using the system call to execute it, this code works with a block. It stores the current execution context, switches to the target context, then yields to the block ensuring that anything run is run in the target's context. When done, forced through the ensure clause, it puts the context back to where it's supposed to be and continues on its merry way.

This has a couple of benefits. First, passing in a block to process is far more Ruby-like than evaluating strings with function calls. Second, it permits more than one statement to be executed at a time in the new context and avoids all the swapping in and out of contexts.

There is a drawback, however, for my purposes (I wrote this, recall, specifically to execute external utilities in arbitrary permissions). Now I have to make explicit calls to system in my passed-in block. This, to me, isn't that great a problem. If it annoys me, I'll go digging for a Ruby-like solution to that one as well. (Note the supreme confidence that there is a convenient way in Ruby to do this....)

module UserWrapper

class Context

def initialize username, path = nil
raise SecurityError , "must be used from root privileges", caller if Process.euid != 0
require 'etc'
@path = path
@username = username
@userinfo = Etc.getpwnam(username)
end

def to_s
"username: #{@username}\nuid: #{@userinfo.uid}, gid: #{@userinfo.gid}\npath: #{@path}"
end

def execute
store
switch_context
begin
yield
ensure
restore
end
end

private

def store
@cuid = Process::Sys.geteuid
@cgid = Process::Sys.getegid
@cpath = ENV["PATH"]
end

def switch_context
Process::Sys.setegid @userinfo.gid
Process::Sys.seteuid @userinfo.uid
ENV["PATH"] = @path if @path
end

def restore
Process::Sys.seteuid @cuid
Process::Sys.setegid @cgid
ENV["PATH"] = @cpath
end

end

end

2007-03-23

Cooking

I have need in my Ruby/Rake script to switch user identities and manipulate the path used by the system when I execute stuff. Solving this problem led me to an epiphany about living in China: it really sucks, sometimes, that I don't have access to the books I used to eat, drink and breathe practically.

The book I really needed was this one and now it's winging its way here through the joint efforts of Canada Post and China Post. (I hope O'Reilly can forgive me for having downloaded an illicit PDF version of it in the interim until I get my physical copy. If they can't, I'll try and live with the bitter disappointment. Copyright infringement doesn't really bother me if it's a stop-gap and since I already own the book, just don't have possession, I don't even think there's a ghost of moral wrong about this.)

Looking over my previous attempts to solve this problem just leaves me shaking my head. Here's a brief history so others will learn the major lesson of today's blog: always find someone else's solutions or ideas before trying out your own. And if your language has a half-decent cookbook-style book? Get it.

"Solution" #1 was an embarrassing attempt to use "sudo" to switch users at key points. Early on in making an LFS build this is fine. Most of what you're doing is system administration stuff anyway: mounting partitions, setting up user accounts, etc.

This all begins to fall apart, however, after that stage. You see you have to set up some environment variables for builds and adjust paths so that as tools get built they are in turn used to build remaining tools. And you have to execute it from a specific user account. This rapidly snowballs out of control. I found myself making code that put environments into shell scripts then manually invoking bash using the . command to get said environment all so that I could then run commands. With sudo and its attendant password mania.

Scratch one solution.

"Solution" #2 wasn't quite as embarrassing, but too managed to become difficult to trace execution of the process. Basically the Rakefile was built into several parts with one part executed from root using sudo as expected which would set up the environment. It would also set up certain key files in the created LFS account that set the appropriate environment variables. Its final task was to copy itself into that LFS account's space and add a call to Rake with a second target into the .bashrc. Then a single "sudo -i" call got made which then executed Rake as appropriate and continued the build.

This was much cleaner than the original version and much less full of the password mania, but it was still very clunky. I'm not a big fan of this kind of layered script and nor am I a fan of splitting scripts up unnecessarily especially for something as silly as switching users. Still, what's a guy to do?

This is where solution #3 enters. I had heard lots of good buzz surrounding the Ruby Cookbook, but because of my situation have no opportunity to sample it in stores to see if the buzz was warranted. So I broke the law and downloaded a copy of it to look at. It instantly went onto my wish list and became a much-coveted purchase. Upon stumbling over a real-world solution to my problem in it, I upped the ante and had family order me a copy in Canada to ship to me using my sparse $CDN account. (I took the opportunity to get a copy of the Pickaxe while I was at it. Shipping these two items was very nearly as expensive as buying the books in the first place.)

So what was the solution? Well, Etc combined with Process::Sys allows me to do all the user switching and path manipulation I want with a minimum of fuss and bother. In the end I used the following code to do what I wanted with only one sudo call -- the one to invoke Rake in the first place.

module UserWrapper
class Context
def initialize username, path = nil
@path = path
@username = username
@uid,@gid = getuserinfo username
end

def getuserinfo username
require 'etc'
u = Etc.getpwnam(username)
return u.uid, u.gid
end

def to_s
"#{@username}:(u#{@uid},g#{@gid}), p#{@path}"
end

def execute command
store
begin
Process::Sys.setegid @gid
Process::Sys.seteuid @uid
system command
ensure
restore
end
end

def store
@cuid = Process::Sys.geteuid
@cgid = Process::Sys.getegid
@cpath = ENV["PATH"]
ENV["PATH"] = @path if @path
end

def restore
Process::Sys.seteuid @cuid
Process::Sys.setegid @cgid
ENV["PATH"] = @cpath
end
end
end


The end result of all the gyrations is something that looks like this:

u_whatever = UserWrapper::Context.new(
"whatever", "/home/whatever/bin" + ":" + ENV["PATH"])

.
.
.
u_whatever.execute "some-command-that-only-exists-in-whatever-bin"

Each time the resulting execute method is called, the current uid and gid are stored, the target user's uid and gid are put in place (effective, not actual!), the path, if provided, is also stored and modified, and the command finally executed. Everything is then restored. Multiple instances of UserWrapper::Context can be stored and invoked at will. The only caveat is that if this is attempted from a process with insufficient privileges, needless to say the whole thing will not work.

Future enhancements I'm eyeing include the ability to switch contexts for the longer term (i.e. for a series of commands) to avoid the constant swapping of uids, gids and paths. I'm also looking at ways to clean up the path management code so that creating the user context doesn't look quite so fugly. (I'm thinking specifically in terms of :prepend and :append arguments that will just manipulate the current path rather than requiring the user to do it all the time.)

Next barrier? The chroot jail used to build the LFS system. What a headache that's being....