2012-02-03 23:42:15

Cloudera Manager Hadoop Cluster Installation Evaluation

I decided to give Cloudera's Manager software a try, and was pleasantly surprised at how simple it becomes to deploy a substantial Hadoop cluster.

I began by creating an automated kickstart installer for RHEL 6.2 (booting off a custom isolinux image created for this purpose), with all of the required packages, so that from server power on to creating a 20+ node cluster takes less than 15 minutes. The limitation for the number of concurrent node installs is based on network and disk i/o bottlenecks on the deployment server. If you wanted to PXE boot the cluster in a production environment, you would want a bank of servers behind a load balancer, optimally.

Once the Manager is installed on the master node, you simply log into the administration webpage, and from there, add all of the hosts to deploy the cluster on. One nice discovery was that it takes advantage of regular expressions for host names or IP addresses, so you can literally create a cluster containing hundreds of nodes with a trivial amount of effort.

Once the software is deployed, you can select the roles for each of the servers. It's an incredibly painless deployment. That being said, it is not without its flaws.

One of the primary flaws is that all of the configuration and log files are in non-standard locations, and are split in non-standard ways. It's obvious from the way that the files are arranged that it simplifies programmatic deployment. It also makes it a bit harder for a human who is used to standard Hadoop deployments to figure out where everything is located.

And finally, I discovered a bug with one of the packaged software products, Oozie. One of the resource files, oozie-bundle-0.1.xsd contains an invalid regular expression on line 22. I haven't tracked down the behavior, but for some reason JDK 1.6.30 will parse that invalid regex, but JDK 1.7U2 will exit with errors. Naturally, I was running JDK 1.7U2, so it took me a little extra time to debug the problem.

Overall, I quite liked Cloudera's Manager. It's certainly one of the better cluster deployment products I've seen.

Tags:   hadoop     |    Perm Link:   Cloudera Manager Hadoop Cluster Installation Evaluation



2010-09-08 07:28:20

Calculating network subnetting with Python

I created this program last year just as an exercise to make sure that I understood the mathmatical processes behind IP subnetting, and could create a network calculator based off that information.

The code below isn't meant to be pretty ... If I spent a little time on it, I could modularize, simplify, and comment the heck out of it, but this code is a byproduct of the network subnetting task that I set for myself, not the end result.

The program will randomly generate an IP and subnet, then generate a quiz for the following questions:

  • Class
  • Increment
  • Netmask
  • IP Address in binary
  • Netmask in binary
  • Network IP
  • Broadcast IP
  • Number of subnets
  • Number of hosts
#!/bin/env python
#################################################################
# Who: James Conner
# When: Feb 24, 2009
# What: pysub
# Version: 0.0.1
# Why: Test subnetting for Cisco exams
#################################################################
# Updates:
# 0.0.1:James Conner:Feb 24, 2009
#   Initial creation
# 0.0.2:James Conner:Feb 27, 2009
#   Added network and broadcast IPs
#################################################################
# Notes:
# Need error and exception controls
#################################################################

#################################################################
# Modules
#################################################################
import optparse
import sys, os
import string
import random
import datetime


#################################################################
# Global Variables
#################################################################
stamp = datetime.datetime.now()


#################################################################
# Classes
#################################################################


#################################################################
# Functions
#################################################################
def int2bin(n, count=8):
  return "".join([str((n >> y) & 1) for y in range(count-1, -1, -1)])

def gen_IP():
  oct1=random.randrange(0,223)
  oct2=random.randrange(0,255)
  oct3=random.randrange(0,255)
  oct4=random.randrange(0,255)
  while oct1 == 127:
    oct1=random.randrange(0,223)
  ipaddr=str(oct1)+"."+str(oct2)+"."+str(oct3)+"."+str(oct4)
  ipaddr_list=[oct1,oct2,oct3,oct4]
  ipaddr_list_binary=[int2bin(ipaddr_list[0]), int2bin(ipaddr_list[1]), int2bin(ipaddr_list[2]), int2bin(ipaddr_list[3])]
  ipaddr_binary=ipaddr_list_binary[0]+"." +ipaddr_list_binary[1]+"." +ipaddr_list_binary[2]+"." +ipaddr_list_binary[3]

  if oct1 <= 126:
    netclass="A"
  elif oct1 <= 191:
    netclass="B"
  elif oct1 <= 223:
    netclass="C"
  else:
    print("Determining netclass failed")

  randmask=str(256-(2**random.randrange(0,8)))

  if netclass == "C":
    while randmask=="255":
      randmask=str(256-(2**random.randrange(0,8)))
    netmask_temp=[255,255,255,randmask]
  elif netclass == "B":
    a=random.randrange(0,1)
    if a == 0:
      while randmask=="255":
        randmask=str(256-(2**random.randrange(0,8)))
      netmask_temp=[255,255,255,randmask]
    elif a == 1:
      netmask_temp=[255,255,randmask,0]
    else:
      print("netclass_temp B is out of bounds")
  elif netclass == "A":
    a=random.randrange(0,2)
    if a == 0:
      while randmask=="255":
        randmask=str(256-(2**random.randrange(0,8)))
      netmask_temp=[255,255,255,randmask]
    elif a == 1:
      netmask_temp=[255,255,randmask,0]
    elif a == 2:
      netmask_temp=[255,randmask,0,0]
    else:
      print("netclass_temp A is out of bounds")
  else:
    print("No netclass corresponding")
  netmask_list=[netmask_temp[0],netmask_temp[1],netmask_temp[2],netmask_temp[3]]
  netmask=str(netmask_list[0])+"." +str(netmask_list[1])+"." +str(netmask_list[2])+"."+ str(netmask_list[3])
  netmask_list_binary=[int2bin(int(netmask_list[0])), int2bin(int(netmask_list[1])), int2bin(int(netmask_list[2])), int2bin(int(netmask_list[3]))]
  netmask_binary=str(netmask_list_binary[0])+"." +str(netmask_list_binary[1])+"." +str(netmask_list_binary[2])+"." +str(netmask_list_binary[3])
  increment=256-int(randmask)
  netip_list=[int(ipaddr_list[0])& int(netmask_list[0]),int(ipaddr_list[1])& int(netmask_list[1]),int(ipaddr_list[2])& int(netmask_list[2]),int(ipaddr_list[3])& int(netmask_list[3])]
  broadcastip_mask_list=[((int(ipaddr_list[0])| int(netmask_list[0]))-255)*-1,((int(ipaddr_list[1])| int(netmask_list[1]))-255)*-1,((int(ipaddr_list[2])| int(netmask_list[2]))-255)*-1,((int(ipaddr_list[3])| int(netmask_list[3]))-255)*-1]
  broadcastip_list=[int(ipaddr_list[0])+ int(broadcastip_mask_list[0]), int(ipaddr_list[1])+ int(broadcastip_mask_list[1]), int(ipaddr_list[2])+ int(broadcastip_mask_list[2]), int(ipaddr_list[3])+ int(broadcastip_mask_list[3])]


  broadcastip=str(broadcastip_list[0])+"."+ str(broadcastip_list[1])+"." +str(broadcastip_list[2])+"." +str(broadcastip_list[3])
  netip=str(netip_list[0])+"." +str(netip_list[1])+"." +str(netip_list[2])+"." +str(netip_list[3])


  if netclass == "A":
    subnets=(2**(str(netmask_binary).count("1")-8))
  elif netclass == "B":
    subnets=(2**(str(netmask_binary).count("1")-16))
  elif netclass == "C":
    subnets=(2**(str(netmask_binary).count("1")-24))
  else:
    print("subnets is messed up")

  hosts=(2**(str(netmask_binary).count("0"))-2)

  print("-"*64)
  print("IP: "+str(ipaddr)+"/"+str(str(netmask_binary).count("1")))
  print("-"*64)

  answer_class = raw_input("Class: ")
  if answer_class == netclass:
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(netclass))

  answer_increment = raw_input("Increment: ")
  if str(answer_increment) == str(increment):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(increment))

  answer_netmask = raw_input("Mask: ")
  if str(answer_netmask) == str(netmask):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(netmask))

  answer_ipaddr_binary = raw_input("IP Binary: ")
  if str(answer_ipaddr_binary) == str(ipaddr_binary):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(ipaddr_binary))

  answer_netmask_binary = raw_input("Mask Binary: ")
  if str(answer_netmask_binary) == str(netmask_binary):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(netmask_binary))

  answer_netip = raw_input("Network IP: ")
  if str(answer_netip) == str(netip):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(netip))

  answer_broadcastip = raw_input("Broadcast IP: ")
  if str(answer_broadcastip) == str(broadcastip):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(broadcastip))

  answer_subnets = raw_input("Subnets: ")
  if str(answer_subnets) == str(subnets):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(subnets))

  answer_hosts = raw_input("Hosts: ")
  if str(answer_hosts) == str(hosts):
    print ("Correct!")
  else:
    print ("Wrong! The answer is: "+str(hosts))



#################################################################
# Option Parser
#################################################################
parser = optparse.OptionParser(version = "0.0.2")
parser.set_defaults(mode="advanced")
(opts, arg) = parser.parse_args()


#################################################################
# Program Execution
#################################################################
if __name__ == '__main__':
    gen_IP()

Tags:   python     |    Perm Link:   Calculating network subnetting with Python



2009-12-21 20:34:51

Craig Ferguson has problems with airlines, too

Tags:   travel     |    Perm Link:   Craig Ferguson has problems with airlines, too



2009-07-27 09:04:44

Kill shell session using pkill

Every once in a while, you need to include the ability for a shell script to commit seppuku, and to kill the shell that spawned it. This is sometimes needed if you're running a program which is known to hang the shell, for instance.

Another common usage, if you're a BOFH, is to replace a common executable file with a shell script that terminates the user's shell that executed the script. I'm making no judgments! :o)

/usr/bin/pkill -9 -s 0 -o

The command breaks down as:

  • pkill: base command
  • -9: kill signal for "terminate & exit immediately"
  • -s 0: session 0, which is pkill's own ID
  • -o: oldest process, which will be the shell

Tags:   linux     |    Perm Link:   Kill shell session using pkill



2009-06-20 01:00:21

One line command to setup SSH keypairs

Setting up SSH keypairs isn't a terribly complex task, but it can be tedious if you have a lot of users who aren't Linux or SSH savvy. Simplifying a task to a simple command is always a nice goal when operating in such an environment.

So, keeping that in mind, here's a simple "one liner" alias that can be placed into auto executing profiles (such as /etc/profile, /etc/bashrc, etc). When a user runs the command, it will generate a custom RSA keypair, of 2048bit strength, with a null password. It copies the pub key first into the authorized_keys file on the local server, and then SSH's into the remote server to place it there, as well. Make sure to change "servername" to the name of your actual server!

Once this is done, the user is able to SSH from the local server to the remote without being prompted for a password. This is obviously useful for things like batched and automated processes.

Naturally, this type of passwordless authentication is a security concern, so use it sparingly and wisely.

export auto-keypairs='cd ~ ; mkdir ~/.ssh; ssh-keygen -t rsa -b 2048 -N "" -C "Auto Keypairs" -f ~/.ssh/rsa_auto ; cat ~/.ssh/rsa_auto.pub >> ~/.ssh/authorized_keys ; chmod 600 ~/.ssh/authorized_keys ; ssh servername "mkdir ~/.ssh ; echo `cat .ssh/rsa_auto.pub` >> .ssh/authorized_keys ; chmod 600 ~/.ssh/authorized_keys"'

Tags:   linux     |    Perm Link:   One line command to setup SSH keypairs



2009-03-04 21:47:01

Fountain pen: Pilot Vanishing Point 2008 Limited Edition

While I was in Hong Kong last October, I picked up a Pilot Vanishing Point Capless 2008 Limited Edition fountain pen from Feng Yuan Pens in Star House, Kowloon.

Namiki Vanishing Point Capless 2008 LE (purple)

Right now I've got it loaded up with Diamine Violet, which is a pretty close match to the color of the pen barrel. This will probably be my last VP purchase for a while, since I now have 3 of them. The next step is to order some customized nibs from Richard Binder.

Tags:   pens     |    Perm Link:   Fountain pen: Pilot Vanishing Point 2008 Limited Edition



2009-02-25 20:25:01

Excellent video on the credit crisis

Here's an excellent video I found, which explains the credit crisis in plain, easy to understand language.


The Crisis of Credit Visualized from Jonathan Jarvis on Vimeo

Tags:   news     |    Perm Link:   Excellent video on the credit crisis



2009-02-14 02:13:25

Func and CertMaster

One of the major complications with dealing with large server farms is the easy distributing and executing of arbitrary commands.

The most common solution for this issue is SSH key pair exchanges. I've built a few solutions using that design, and it works quite well ... to a point. Things like key revocation, meta-character handling, error and exception handling, and so forth are a bit of a pain to deal with in that sort of solution. Scale it up to a few hundred servers, and now you have a problem manging the whole thing. Working within a structured language like Python, using the Paramiko SSH module, you can largely mitigate most of those issues, but it's not perfect. Your keypair administration is still overly complicated. Enter Func and CertMaster.

CertMaster is an SSL certificate server ... long story short, it handles the authentication piece that replaces the SSH keypairs. Func (Fedora Unified Network Controller) is a program which grabs the server SSL cert from the CertMaster server, and allows you to run arbitrary commands through SSL encrypted channels. Overall, that doesn't get you much more than the old way of using the SSH keypairs ... but there's a twist.

Func includes a Python API, simplifying the entire process of executing arbitrary commands, and also enables the ability to easily add modules for frequently used processes, or very large processes. Couple that into the ability to execute asynchronously, and you've got an interesting solution. Fun stuff.

Tags:   python     |    Perm Link:   Func and CertMaster



2009-02-14 00:21:35

LaCie 5big NAS device

I've been on the lookout for a decent NAS product with at least 4 hard drive slots. The 5big device might actually fit the bill. The only downside is that it doesn't come in a diskless version. That means you have to overpay for the drives, of course.

Some nice features include:

  • Active Directory integration
  • 2 x 1GB Ethernet ports
  • CIFS, FTP, HTTP and Bonjour
  • RAID 6
  • Integrated Bittorrent client
  • Integrated self-monitoring
  • 1 x 2.0 USB for external drives
  • 3 x eSATA for external drives

Tags:   gadgets     |    Perm Link:   LaCie 5big NAS device



2009-02-13 23:52:13

My "holy grail" pen

This pen is absolutely gorgeous. Silver guilloche overlay in a flame pattern with a red celluloid section and silver trim. I wish it wasn't a limited edition, so it was easier to actually find them. The pen was actually issued in 1999 for Aurora's 80th Anniversary. I'm hoping that their 90th will be as equally sexy.

Aurora 80th Anniversary Limited Edition

Tags:   pens     |    Perm Link:   My "holy grail" pen



2009-02-13 21:50:49

Compressed air jetpack

I almost hurt myself laughing at this video. Apparently, he went about 80 feet horizontally, and 30 feet vertically. Not to mention the 200mph launch speed!

Tags:   humor     |    Perm Link:   Compressed air jetpack



2009-01-18 15:02:12

Visconti Van Gogh Maxi in Cappuccino

Here's a little pen I picked up some time ago ... the Visconti Van Gogh Maxi, in Cappuccino colors with Gold trim.

The pen is really an exceptional writer, and I'm evaluating whether or not I would prefer a medium or a fine nib. It currently has a medium, which is a tad thicker than I normally prefer, but the trade off is that the nib is fantastically smooth and has a huge sweet spot, in comparison to the Asian fine nibs I've been using lately.

Visconti Van Gogh in Cappuccino

Tags:   pens     |    Perm Link:   Visconti Van Gogh Maxi in Cappuccino



2009-01-17 17:46:15

Using Python to decode SAS passwords

One of the SAS recommended methods for dealing with scripted processes is to use a clear text password in SAS 9.1. Please don't do that.

The other method promotes security by using an encoded password. Unfortunately, the password can be easily reversed because it uses a trivial encoding mechanism.

Here is an example, using SAS, to generate an encoded password for usage in SAS scripts:

proc PWENCODE in="flummoxedpygmy";
run;
{sas001}Zmx1bW1veGVkcHlnbXk=

And now, to decode it using Python:

[root@navi ~]# python
Python 2.5.1 (r251:54863, Jun 15 2008, 18:24:56)
[GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import base64
>>> coded_pwd = "Zmx1bW1veGVkcHlnbXk="
>>> decoded_pwd = base64.b64decode(coded_pwd)
>>> print(decoded_pwd)
flummoxedpygmy

As it stands, the method that SAS uses for password encoding leaves the SAS accounts vulnerable to being hacked by reversing that encoding, if the script files can be read by users. One way hashes, with salts, would have provided better security, and would be easy to implement.

Tags:   python     |    Perm Link:   Using Python to decode SAS passwords



2008-11-10 23:27:27

Anniversary Vacation In Mexico

Meredith and I decided to take a week long vacation to Mexico this year to celebrate our second year anniversary. Meredith like it so much, she's talking about buying a place down there permanently!

Chemuyil







Akumal



Tulum













Ek Balam











Yal Ku





Chichen Itza







Ik Kil



Resort





Tags:   travel     |    Perm Link:   Anniversary Vacation In Mexico



2008-10-09 19:55:31

Pictures of Hong Kong

Hong Kong was a blast! There was so much to do and see that there was no way to fit it into a 2 week trip. I'm looking forward to going again, but next time with Meredith.

Kowloon



















Central









Ngong Ping



















Diamond Hill































Shau Kei Wan





Tags:   travel     |    Perm Link:   Pictures of Hong Kong



2008-09-20 19:16:04

Preparing for Hong Kong

I've been getting ready for my upcoming trip to Hong Kong, and as part of the process, I've been setting up a Google "My Maps" of the places I want to visit.

I decided to go on this trip just a week ago, since I found a fantastic fare on Delta. Overall, I was able to pay for the ticket, and 8 nights in a Marriott for less than the airplane ticket to Saigon, which is where I was originally headed.

One of the things I'm very excited about is the chance to visit some Hong Kong tailors personally. I've ordered from Hemrajani Brothers before, but this will be the first chance I've had to actually talk directly to the tailors, rather than the traveling salesman. I've marked down a few tailors on the map that I'll be visiting on the first day that I'm there (leaving plenty of time for second or third fittings during the rest of the trip), and I'll post my reviews afterwards.


view the full sized map

Tags:   travel     |    Perm Link:   Preparing for Hong Kong



2008-09-01 16:00:13

England - August 2008 for Meredith's Birthday

Meredith and I went over to London, England this year for her birthday. We saw some sights, went to a couple shows, and ate at some excellent restaurants.

Hampton Court







Westminster Abbey





Dining






Tags:   travel     |    Perm Link:   England - August 2008 for Meredith's Birthday



2008-08-21 23:15:13

Hash fields in a CSV file with Python and SHA256

I had a request recently where someone asked for a quick and dirty script that would take a CSV file, and parse certain fields for hashing. I decided to use python due to the nice speed of the hashlib module (and because I simply like python!).

If you look at the code, it doesn't contain two things ... comments and error checking. I know, I know. That's coming in the next version. Remember, I was asked for "quick and dirty"!

One thing to note is that the script can handle multiple field arguments. So, for example, you can use the following argument: "-f 2 -f 3 -f 8" and it will parse the 3rd, 4th and 9th field (counting begins at zero :)

The other thing to notice is that there is a --md5 option. I was asked to add that feature since the SHA256 output is extremely long and takes up a lot of space in a database when you're talking a few billion rows. The --md5 option takes the SHA256 output and performs an MD5 hash on it, reducing it to a smaller, but still unique, string.

#!/bin/env python
#################################################################
# Who: James Conner
# When: June 16, 2008
# What: csv_hash.py
# Version: 1.0.1
# Why: Encrypt fields within a CSV file
#################################################################
# Updates:
# Ver:Who:When:Why
# 0.0.1:James Conner:Jun 16 2008:Initial creation
# 0.0.2:James Conner:Jul 07 2008:Added multi field hashing
# 1.0.0:James Conner:Aug 21 2008:Added MD5 output of SHA256 hash
# 1.0.1:James Conner:Aug 21 2008:Fixed field check
#################################################################

#################################################################
# Import Modules
#################################################################
import os
import sys
import string
import hashlib
import csv
from optparse import OptionParser

#################################################################
# Global Variables
#################################################################

#################################################################
# Option Parser
#################################################################
parser = OptionParser(version = "1.0.1")

parser.add_option('-d','--delimiter',
        dest='delimiter_info',
        default=',',
        metavar='DELIMIT',
        help=('The delimiter. Default is \',\'.'))

parser.add_option('-i','--infile',
        dest='infilename_info',
        default='',
        metavar='INFILENAME',
        help=('Name of the input file'))

parser.add_option('-f','--field',
        dest='field_info',
        action='append',
        type='int',
        metavar='FIELDNUM',
        #default='0', # int does not allow a default value
        help=('Which field to encrypt, default is 0'))

parser.add_option('-o','--outfile',
        dest='outfilename_info',
        default='outfile.csv',
        metavar='OUTFILENAME',
        help=('Name of the output file'))

parser.add_option('--md5',
        dest='md5_info',
        action='store_true',
        default=False,
        metavar='MD5',
        help=('Use MD5 against the SHA256 hash.'))

(opts, arg) = parser.parse_args()

#################################################################
# Functions
#################################################################

def check_options(option,option_name):
        if not option:
                print "ERROR: %s variable has not been assigned!" % option_name
                sys.exit(10)

def perf_hash (DATA,MD5):
        if MD5 is True:
                TEMP_DATA = hashlib.sha256(DATA).hexdigest().strip()
                return (hashlib.md5(TEMP_DATA).hexdigest())
        else:
                return(hashlib.sha256(DATA).hexdigest())

def csv_file (FILENAME,DELIMITER,FIELDNUM,OUTFILE,MD5FLAG):
        r = csv.reader(open(FILENAME, 'r'), dialect='excel', delimiter=DELIMITER)
        w = csv.writer(open(OUTFILE, 'w'), dialect='excel', delimiter=DELIMITER)
        for rows in r:
                for f in FIELDNUM:
                        rows[int(f)] = perf_hash(rows[int(f)],MD5FLAG)
                w.writerow(rows)
        sys.exit(0)

#################################################################
# Program Execution
#################################################################

check_options(opts.infilename_info, "infilename_info") check_options(opts.field_info, "field_info")

csv_file(opts.infilename_info, opts.delimiter_info, opts.field_info, opts.outfilename_info, opts.md5_info)

Tags:   python     |    Perm Link:   Hash fields in a CSV file with Python and SHA256



2008-05-22 22:41:31

American Airlines fee increases

It's amazing the amount of fuss and furor that is taking place due to American Airline's decision to charge $15 for the first bag checked on each flight; yet hardly anyone has mentioned the other outrageous fee hikes that accompanied that decision. For example, a 50% increase in the cost to re-book all flights. Want to travel with Rover? Another 50% fee increase. Unaccompanied minor? Tack on another 33%.

I think people are taking the wrong tact about the baggage, to be honest. If AA wants to charge for the first bag checked, then by all means, they should. However, they should also be held to the commitment of delivering said bag. According to AA, they have no intention of issuing refunds if they lose your luggage. I'm still trying to figure out the logic of that one. Why should people pay money if AA can't deliver the goods? In the name of good customer service, they should make it standard policy to issue a refund in the case of lost luggage.

Tags:   news     |    Perm Link:   American Airlines fee increases



2008-05-22 08:22:04

Chroot SFTP using OpenSSH

The open source community has been in rather desperate need of a good, clean method of chrooting SFTP sessions, but at the same time, denying shell access. In the past, people have gotten around this lack by hacking the SSH or SFTP binaries, or creating a restricted shell by various means. The basic problem with these methods is a lack of standardization and integration. Fortunately, the OpenSSH team (which is part of the OpenBSD Project), have included the ability to do just that in the recent versions of OpenSSH. And what's more, it can do it without all the /dev, /etc and shell requirements typically required for a chroot environment.

This guide will go through the entire process of setting up a chroot (aka jailed) SFTP environment for Fedora, from beginning to end.

The major benefits of utilizing this method of chrooting SFTP:

  • Denies shell access
  • Simplified installation/setup/maintenance
  • Forces users to use an encrypted session for file transfers
  • Can be installed as a separate sshd binary, leaving your current SSH configuration intact

While this method of chrooting is better than what was previously available, it isn't perfect:

  • Utilizes local (or potentially LDAP) accounts, instead of an account file, ala many FTP servers
  • The SFTP binary is tied to the SSH binary, and can't be upgraded independently
  • Doesn't allow SCP (SCP requires shell access, which this method blocks)

Notes about this guide:

  • This guide is for Fedora Core, but can be used for other distros, such as Debian or Unbuntu. Just change the software installers as required.
  • The current version of OpenSSH at this point in time is 5.0p1 (the "p" denotes that it is compatible with Operating Systems other than OpenBSD, such as Linux or Solaris), and was released on April 3rd, 2008. This is the version that will be used for this guide.

Configure binaries:

Configure the binaries so that it will not install over the existing SSH installation:
wget ftp://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-5.0p1.tar.gz
tar zxpvf openssh-5.0p1.tar.gz
cd openssh-5.0p1
./configure --with-tcp-wrappers --prefix=/opt/sftponly --sysconfdir=/etc/sftponly
If the configuration fails due to lack of tcp wrappers, run the following command
yum install -y tcp_wrappers\*

Compile the binaries:

Perform the build and then install it:
make
make install

Configure the /etc/sftponly/sshd_config file.

Modify the following variables:
Port 2222
ServerKeyBits 2048
AllowTcpForwarding no
TCPKeepAlive no
PidFile /var/run/sftponly.pid
Remove the following lines at the end of the config file:
Subsystem sftp /opt/sftponly/libexec/sftp-server
Add the following lines:
Subsystem sftp internal-sftp
Match Group sftponly
        ChrootDirectory %h
        ForceCommand internal-sftp
        AllowTcpForwarding no

Daemon setup (to prevent the normal ssdh from getting squashed)

Rename the new sshd binary to sftponly:
mv /opt/sftponly/sbin/sshd /opt/sftponly/sbin/sftponly
Create the sftponly init.d file:
cp -pr /etc/init.d/sshd /etc/init.d/sftponly
Edit /etc/init.d/sftponly and replace the following variables:
prog="sftponly"
KEYGEN=/opt/sftponly/bin/ssh-keygen
SSHD=/opt/sftponly/sbin/sftponly
RSA1_KEY=/etc/sftponly/ssh_host_key
RSA_KEY=/etc/sftponly/ssh_host_rsa_key
DSA_KEY=/etc/sftponly/ssh_host_dsa_key
PID_FILE=/var/run/sftponly.pid
Fix all of the subsys references:
sed -i -e 's/\/var\/lock\/subsys\/sshd/\/var\/lock\/subsys\/sftponly/g' /etc/init.d/sftponly
Cleanup the status message:
sed -i -e 's/openssh-daemon/sftponly/g' /etc/init.d/sftponly
Now generate the link files for the rc#.d directories:
ln -s /etc/init.d/sftponly /etc/rc0.d/K25sftponly
ln -s /etc/init.d/sftponly /etc/rc1.d/K25sftponly
ln -s /etc/init.d/sftponly /etc/rc2.d/S55sftponly
ln -s /etc/init.d/sftponly /etc/rc3.d/S55sftponly
ln -s /etc/init.d/sftponly /etc/rc4.d/S55sftponly
ln -s /etc/init.d/sftponly /etc/rc5.d/S55sftponly
ln -s /etc/init.d/sftponly /etc/rc6.d/K25sftponly

Group setup

Install the sftponly group with a specific GID:
groupadd -g 999 sftponly

Chroot filesystem setup

Make the chroot filesystem, and modify ownership and permissions:
mkdir /chroot
chown root:root /chroot
chmod 755 /chroot

User setup

Create the user, temporarily allowing shell access:
useradd -m -r -N -d /chroot/twinkles -s /bin/bash -g 999 -c "Chroot_User twinkles" twinkles
passwd twinkles
su - twinkles
As the user "twinkles", not root, perform the following commands:
/opt/sftponly/bin/ssh-keygen -b 2048 -t rsa
/opt/sftponly/bin/ssh-keygen -b 1024 -t dsa
touch ~/.ssh/authorized_keys
mkdir -p ~/incoming
mkdir -p ~/outgoing
exit
As root, change the shell to /sbin/nologin, and clean up permissions:
usermod -s /sbin/nologin twinkles
chown root:root /chroot/twinkles
chmod 755 /chroot/twinkles

SSH key authentication

If you want to test the account without having to use passwords:
cat ~/.ssh/id_rsa.pub >> ~twinkles/.ssh/authorized_keys

Ladies and Gentlemen, start your daemons!

Start the daemon:
[root@navi ~]# service sftponly status
sftponly is stopped

[root@navi ~]# service sftponly start
Starting sftponly:        [ OK ]
Verify the daemon is running and attached to the correct TCP port:
[root@navi ~]# service sftponly status
sftponly (pid 11729) is running...

[root@navi ~]# ps -ef | grep sftponly
root        11729        1 0 08:57 ?        00:00:00 /opt/sftponly/sbin/sftponly

[root@navi ~]# netstat -pan | grep 2222
tcp        0        0 0.0.0.0:2222        0.0.0.0:*        LISTEN        11729/sftponly

Testing

With the configuration complete and the daemon started, you obviously would want to test it. Here are examples of what you should see:

[root@navi openssh-5.0p1]# sftp -oPort=2222 twinkles@localhost Connecting to localhost...
sftp>

sftp> pwd
Remote working directory: /

sftp> ls -l
drwxr-xr-x        2 490        999        4096 May 22 02:32 incoming
drwxr-xr-x        2 490        999        4096 May 22 02:32 outgoing

sftp> put sshd.c
Uploading sshd.c to /sshd.c
Couldn't get handle: Permission denied

sftp> cd incoming

sftp> pwd
Remote working directory: /incoming

sftp> put sshd.c
Uploading sshd.c to /incoming/sshd.c
sshd.c                100% 57KB 56.7KB/s 00:00

sftp> exit

[root@navi openssh-5.0p1]# scp --port=2222 sshd.c twinkles@localhost:/
This account is currently not available.

[root@navi openssh-5.0p1]# cd ~twinkles/incoming/

[root@navi incoming]# ls -AFlh
total 64K
-rw-r--r-- 1 twinkles sftponly 57K 2008-05-22 08:44 sshd.c

A couple of important things to note in this example:

  • Make sure to SFTP to port 2222, since that's the TCP port your sftponly binary is configured to, in /etc/sftponly/sshd_config!
  • The user cannot modify the "root" directory. This is because the directory is actually owned by the user root. The user can, however, modify the sub-directories (incoming, outgoing) because he does own those.
  • The scp program won't work, because the user's shell is set to /sbin/nologin.
  • When inside the sftp shell, notice that an ls -l shows UIDs and GIDs, instead of names. This is due to the chroot not having access to the /etc filesystem.

And there you go ... a nicely chrooted SFTP environment.

Tags:   linux     |    Perm Link:   Chroot SFTP using OpenSSH



2008-04-26 09:16:25

Create a bootable Fedora Live image on a USB stick

Every once in a while, I need to haul out a rescue disk for RedHat to fix an issue (usually something like trying to access failed disk systems). Inevitably, I can't find my rescue disk when I actually need it, so I end up having to wait for a new download to finish.

One of the nice features in the recent versions of Fedora is the inclusion of a small script called livecd-iso-to-disk. Just like it says, it creates a Live image on a disk (a USB stick/key/drive/whatever is a removable disk). Joy! Unlike most other image to disk procedures, this is non-destructive to pre-existing data on the disk. In other words, you don't have to dedicate a USB key just for a Live image.

# yum install livecd-tools -y
livna 100% |=========================| 2.1 kB 00:00
fedora 100% |=========================| 2.1 kB 00:00
updates 100% |=========================| 2.3 kB 00:00
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package livecd-tools.i386 0:013-1.fc8 set to be updated
--> Processing Dependency: syslinux for package: livecd-tools
--> Processing Dependency: isomd5sum for package: livecd-tools
--> Running transaction check
---> Package isomd5sum.i386 0:11.3.0.50-2 set to be updated
---> Package syslinux.i386 0:3.36-7.fc8 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================
Package Arch Version Repository Size
=================================================================
Installing:
livecd-tools i386 013-1.fc8 fedora 48 k
Installing for dependencies:
isomd5sum i386 11.3.0.50-2 fedora 149 k
syslinux i386 3.36-7.fc8 fedora 678 k

Transaction Summary
=================================================================
Install 3 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

Downloading Packages:
(1/3): livecd-tools-013-1 100% |=========================| 48 kB 00:00
(2/3): syslinux-3.36-7.fc 100% |=========================| 678 kB 00:01
(3/3): isomd5sum-11.3.0.5 100% |=========================| 149 kB 00:01
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: isomd5sum ######################### [1/3]
Installing: syslinux ######################### [2/3]
Installing: livecd-tools ######################### [3/3]

Installed: livecd-tools.i386 0:013-1.fc8
Dependency Installed: isomd5sum.i386 0:11.3.0.50-2 syslinux.i386 0:3.36-7.fc8
Complete!

Next, check to make sure that the USB stick is bootable.

# parted /dev/sdb print
Model: Imation Flash Drive (scsi)
Disk /dev/sdb: 1062MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 1062MB 1062MB primary fat16

Information: Don't forget to update /etc/fstab, if necessary.

Okay, so our USB stick isn't bootable; easy enough to fix!

# parted /dev/sdb
GNU Parted 1.8.6
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Imation Flash Drive (scsi)
Disk /dev/sdb: 1062MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 1062MB 1062MB primary fat16

(parted) toggle 1 boot
(parted) print
Model: Imation Flash Drive (scsi)
Disk /dev/sdb: 1062MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 1062MB 1062MB primary fat16 boot

(parted) quit
Information: Don't forget to update /etc/fstab, if necessary.

Now we're ready to actually copy the image to the disk.

# livecd-iso-to-disk Fedora-8-Live-i686.iso /dev/sdb1
Copying live image to USB stick
Updating boot config file
Installing boot loader
USB stick set up as live image!

Now you have a bootable drive, without destroying the other data on the USB device. Yay!

Tags:   linux     |    Perm Link:   Create a bootable Fedora Live image on a USB stick



2008-04-18 21:30:50

IronMan vs Linux ... the loser is Paramount!

I am not, by any stretch of the imagination, a comic book fan. Regardless, that doesn't preclude any enjoyment to be had from an afternoon spent at the movie theatre, enjoying the latest efforts based on those indelible icons of Americana.

That being said, one must wonder what Marvel and Paramount Pictures were thinking when they let Matt Fraction, writer for the new "Invincible Iron Man" comic from Marvel, open his mouth regarding the new comic book, prior to the movie premier!

"[Tony Stark, aka IronMan, the Hero]'s the future's pioneer, the Chuck Yeager of human evolution. He's at the forefront of what comes next."

"[Ezekiel Stane, the Villian] has absolutely no loyalty to any sort of law, creed, or credo. He doesn't want to beat Tony Stark, he wants to make him obsolete. Windows wants to be on every computer desktop in the world, but Linux and Stane want to destroy the desktop. He's the open source to Stark's closed source oppressiveness."

Is he intentionally attempting to alienate part of his fan base, or is he just an idiot?

I can honestly say that it is the first time I’ve heard "closed source oppressiveness" described as "the forefront of what comes next". Not to mention "Linux ... want[s] to destroy the desktop".

Whatever. That's a pair of $9.75 tickets and a $30 blu-ray sale lost.

Tags:   personal     |    Perm Link:   IronMan vs Linux ... the loser is Paramount!



2008-04-15 21:26:47

Fun with Patents!

Patent reform has become a major topic of discussion in the tech industry within the last 10 years. The majority of consternation has revolved around Software Patents, and Patent Trolls.

Software Patents have been coming under fire as patents are submitted and accepted for approval that cover basic concepts. One of the first that created major concern was the 1-Click patent issued to Amazon.com in 1999. In a nutshell, here’s the concept: You click on a "Buy Now!" button, and your login details are used to pull back database information containing your Credit Card, address, and so forth. That's it.

Patent Trolls have always existed, but in the past 30 years, "Litigation" has replaced "Innovation" as a profitable business model. If you look at a graph of the number of patents issued per year, dating back to 1850, there is a clear, nearly linear (except during the WWII and Vietnam Eras) progression of patents issued for approximately 120 years.

Then, around the 1970s, the patent system starts going insane ... I don't know if it's simply the number of patents being requested went up, and as a consequence, the numbered issued goes up, or if the same number were submitted, but more passed. I'm sure there are records of that somewhere on the USPTO website. One thing is clear, however: The concept of "Profit by Litigation" really begins to take off, and companies form patent portfolios not only for offense, but defense. The next logical step is to form companies that act strictly as "Patent Speculators". They buy up patents or hire inventors and engineers to create patents for their company, and then wait until they can nail a company for infringement. One of the most infamous examples of this behavior is the NTP vs RIM (Blackberry) case. You know, the one where the innovator paid the litigator $612.5 Million dollars.

So, why this long diatribe for Patent Reform? Because companies actually have DESIGN patents and attempt to stifle innovation based on them!

Monster Cables is one of those companies ... earlier this year, they sent a C&D to Blue Jean Cables, a small company that produces good quality hardware, based on reviews. In a nutshell, Monster claims BJC violates 5 separate patents owned by Monster, and must therefore settle. Apparently, what Monster's lawyers did not realize that BJC's president is a graduate of Univ. of Pennsylvania Law school, with 19 years of litigation practice. My favorite quote: "Not only am I unintimidated by litigation; I sometimes rather miss it."

You can find a copy of BJC’s response to Monster at Audioholics. Stop by and have a look; the response is long, but oh, so worth the read!

Tags:   humor     |    Perm Link:   Fun with Patents!



2008-04-14 22:52:30

401k Retirement Calculator for Excel

Since I've been promoted, I decided to review my 401k contributions. There are some nice online calculators out there, like Bloomberg's, but I prefer being able to see the exact nature of the calculations, which most online calculators obscure.

Enter the 401k Excel Calculator from Vertex42. It's got all the features of the online calculators, but also lets you see behind the curtains. One nice feature is the ability to randomize the returns per year, allowing a slightly more realistic earnings curve, rather than the exponential curve created by an unchanging return rate.

As good as the Vertex42 calculator is, it does make one critical mistake (which it warns you about, but curiously doesn't prevent): It can contribute more than the legal amount in its calculations. The legal amount changes by age (ie, if you're 50 or older, you can contribute 20,500, otherwise you can only contribute 15,500).

So what I've done is modify the original Vertex42 Excel file, and added a few IF statements which limit the contribution amounts to the legal values. If you'd like to download it, here it is: Modified version of the Vertex42 calculator. Just make sure to visit Vertex42 and give them some love!

Tags:   personal     |    Perm Link:   401k Retirement Calculator for Excel



2008-04-06 22:27:35

Promotion!

While I was in London a couple weeks ago, I was given a letter and a handshake. I've been promoted to "Director" grade in dunnhumby, with an official title of "Principle Engineer".

So, what does that mean? Essentially, I have Linux, Fibre and Storage strategy and standardization, Disaster Recovery, Information Lifecycle Management, and blueprint designs for new environments.

All in all, it's a fun gig.

Tags:   personal     |    Perm Link:   Promotion!



2008-04-05 23:27:56

Downtime communications database ...

Yet another little "somethin' somethin'" I've been putting together in my spare time is a downtimes communications database and web portal for declaring downtimes based on servers, clusters, regions, processes, etc. I know there's quite a few products out there already that can do this, but ... 'eh, this is what I do for fun.

The premise is pretty simple ... my company is all about data. It's a market research company, so if you don't have the data, you don't have a product to sell, in a nutshell. This means that any sort of downtimes impacting the availability of data access has to be minimized as much as possible.

The problem, of course, is the complex issue of data ownership. Who is responsible for what data, and needs to be informed of downtimes. By utilizing a communications manager, you can centralize the communications and focus them away from the engineers performing the work. A comms manager, however, doesn't necessarily have the information regarding data ownership for the impacted services. This situation is further exacerbated by phenomenal growth, introducing new data owners at a very high rate.

In this scenario, you need a normalized database with a high grade of referential integrity, linking servers, services, downtime coordinators, communication managers, clusters, regions, and so forth, together into an easy to manipulate fabric.


For this particular project, I chose fairly standard tools ... Python, MySQL and Apache. The Apache/Python interface allows you to select the type of downtime reports that you want to generate. At the moment, it prints out a simple HTML page with all of the requisite data owner and coordinator information, however, the next iteration of the project will include PDF files. The goal is to produce an output which can be attached to Change Management documentation, and included as part of the official change process.

I love this stuff!

Tags:   web, python     |    Perm Link:   Downtime communications database ...



2008-03-30 05:30:34

Boston Dynamics Big Dog

This is a really impressive video, showing how far along actuated, balanced robotics is coming along. The ability to maintain balance has been one of the biggest hurdles in the robotics industry (ASIMO's difficulty with stairs) but obviously they've jumped over that hurdle with the Big Dog!

Tags:   gadgets     |    Perm Link:   Boston Dynamics Big Dog



2008-03-07 20:55:23

Cisco Fibre Switch OID

There is a lot of information out on the web concerning Cisco Ethernet switches, but almost nothing concerning their Fibre switch line. Naturally, that's due in large part to the fact that Cisco is fairly new to the fibre switch scene.

I've been working on setting up RRDTool graphing for some Cisco 9513, 9124 and 9020 fibre switches, but ran into a minor problem. Cisco Fibre switch OIDs do not correspond to Cisco Ethernet switch OIDs for things like CPU and Memory information. They probably made it like that since the Fibre switches actually use a modified Linux kernel and run SANos, rather than IOS. Interface statistics are unchanged, fortunately.

This is where the lack of information regarding Cisco Fibre switches rears its ugly head. Not even on Cisco's own website can you find information about which IOD from the SNMPv2-SMI MIB file matchs CPU percentage used, much less anywhere else on the Internet.

In the end, I was able to discover the information, but it takes an ugly kludge to do so. The trick was to fire up Cisco's Fabric Manager software, which connects to the Fibre switches via SNMP v.3 ... fortunately, only the authentication part of SNMP v.3 is actually encrypted. When the Fibre switches responded, I had Wireshark capture the packets, which contained the OID strings in plain text.

Cisco 9513 Name Cisco 9513 OID
CPU Percent Used 1.3.6.1.4.1.9.9.305.1.1.1.0
Memory Percent Used 1.3.6.1.4.1.9.9.305.1.1.2.0
Bootflash Size 1.3.6.1.4.1.9.9.10.1.1.4.1.1.4.1.1
Bootflash Available 1.3.6.1.4.1.9.9.10.1.1.4.1.1.5.1.1
Processor RAM 1.3.6.1.4.1.9.9.195.1.1.1.1.982
NVRAM Size 1.3.6.1.4.1.9.9.195.1.1.1.2.982
NVRAM Used 1.3.6.1.4.1.9.9.195.1.1.1.3.982

Tags:   fibre     |    Perm Link:   Cisco Fibre Switch OID



2008-03-07 20:25:23

Move over Halo3, make way for Crysis

Halo3 (Xbox) and Crysis (PC) came out at just about the same time, but I was gaming mostly on the Xbox at the time, so I only picked up Halo3.

That apparently turned out to be a mistake. I just picked up Crysis, and it is, in a word, stunning. The visuals are absolutely fantastic, but it comes at a price. My main gaming rig was top of the line two years ago ... AMD 4400X2, 2GB RAM, Geforce 7800GTX, but it's only capable of playing Crysis using the low end video settings. Time for an upgrade ... that ATI 3870x2 is lookin' mighty nice!

Tags:   games     |    Perm Link:   Move over Halo3, make way for Crysis



2008-02-17 20:36:35

Fountain pen: Pilot Vanishing Point in Chrome

Here's the first VP that I purchased a while back. I had problems for a while with the clip interfering with my grip, but now I don't even notice it.

I like the VPs quite a bit since it doesn't have a threaded cap like the rest of my pen collection. By having the nib retract and expose by the button on the end of the pen, you don't have to keep taking off and replacing the cap as you take quick notes. Invaluable for long meetings!


Namiki Vanishing Point Capless in Chrome

Tags:   pens     |    Perm Link:   Fountain pen: Pilot Vanishing Point in Chrome



2008-02-17 20:36:35

Fountain pen: Marlen (dot)com in Autumn

Here's a nice little external piston filling Italian celluloid pen with a Fine steel nib. It's an incredibly smooth writer, and lays down a fairly wet line for a Western Fine nib. Overall, it's one of my favorites, so I use it as a daily writer.


Marlen (dot)com, Autumn
Marlen (dot)com, Autumn

Tags:   pens     |    Perm Link:   Fountain pen: Marlen (dot)com in Autumn



2008-02-17 18:39:54

Fountain pen: Delta Tuareg

Here's a fountain pen that I purchased a while ago: The Delta Tuareg. It's a gorgeous piece of Itallian celluloid, with .925 silver accents. The nib is a broad, and it certainly lays down a thick, wet line. Despite having a substantial ink capacity, a full load of ink lasts for maybe 6 pages or so, due to the incredibly wet line.

My only real complaint about the pen is it occasionally false starts. This can probably be easily fixed by a nibmeister, so I'll hit up one of the pen shows in the East this year to get it adjusted.


Delta Tuareg, 1k Silver Limited Edition

Tags:   pens     |    Perm Link:   Fountain pen: Delta Tuareg



2008-02-17 18:06:11

Fountain pen: Pilot Vanishing Point in Red

For Valentines day, my lovely wife gave me a fountain pen! Specifically, a Namiki/Pilot Vanishing Point (aka Capless), in red. As she said "It's easy shopping for people with an obsession."


Namiki Vanishing Point Capless in Red

Tags:   pens     |    Perm Link:   Fountain pen: Pilot Vanishing Point in Red



2008-02-17 11:37:58

Python snmp example using pysnmp

Here's a simple python program that makes an snmp query against network interface descriptions, and then grabs the physical address of that interface using another snmp call.

#!/usr/bin/env python
import re, os, sys, getopt, string, commands
from pysnmp.entity.rfc3413.oneliner import cmdgen
from pysnmp.entity.rfc3413 import mibvar
from pysnmp.proto import rfc1902

SNMP_HOST='127.0.0.1'
SNMP_RO_STRING='whatever'
SNMP_PORT=161

IOD_sysdescr=[1,3,6,1,2,1,1,1,0]
IOD_descr=[1,3,6,1,2,1,2,2,1,2]
IOD_phyaddr=[1,3,6,1,2,1,2,2,1,6]
IOD_type=[1,3,6,1,2,1,2,2,1,3]
IOD_speed=[1,3,6,1,2,1,2,2,1,5]
IOD_alias=[1,3,6,1,2,1,31,1,1,1,18]
    
def main():
    cmdGen = cmdgen.CommandGenerator()
    errorIndication, errorStatus, errorIndex, varBindTable = cmdGen.nextCmd(
        cmdgen.CommunityData('duofold', SNMP_RO_STRING), cmdgen.UdpTransportTarget((SNMP_HOST, SNMP_PORT)),
        (('IF-MIB', 'ifDescr'),),)

    if errorIndication:
        print errorIndication
    else:
        if errorStatus:
            print '%s at %s\n' % (errorStatus.prettyPrint(),varBindTable[-1][int(errorIndex)-1])
        else:
            for varBindTableRow in varBindTable:
                for oid, val in varBindTableRow:
                    (symName, modName), indices = mibvar.oidToMibName(cmdGen.mibViewController, oid)
                    val = mibvar.cloneFromMibValue(cmdGen.mibViewController, modName, symName, val)
                    #print '%s::%s.%s = %s' % (modName, symName,string.join(map(lambda v: v.prettyPrint(), indices), '.'),val.prettyPrint())
                    INTERFACE=oid.prettyPrint().split('.')[-1:].pop(0)
                    get_attr(INTERFACE,IOD_phyaddr)


def get_attr(interface,attr):
    cmdGen = cmdgen.CommandGenerator()
    IOD_list=attr+[int(interface)]
    IOD=tuple(IOD_list)
    errorIndication, errorStatus, errorIndex, varBinds = cmdgen.CommandGenerator().getCmd(
        cmdgen.CommunityData('edson', SNMP_RO_STRING),cmdgen.UdpTransportTarget((SNMP_HOST, SNMP_PORT)),(IOD))
    if errorIndication:
        print errorIndication
    else:
        if errorStatus:
            print '%s at %s\n' % (errorStatus.prettyPrint(), varBinds[int(errorIndex)-1])
        else:
            for name, val in varBinds:
                if attr==IOD_phyaddr:
                    print '%s = %s' % (name.prettyPrint(), hexify(val))
                else:
                    print '%s = %s' % (name.prettyPrint(), val.prettyPrint())

def hexify(input):
    return ":".join([ '%x'%(ord(s)) for s in input])

main()

Tags:   python     |    Perm Link:   Python snmp example using pysnmp



2008-02-15 15:27:00

Tor gets serious about ebooks

While looking high and low on the internet for a legal ebook of John Scalzi's "Old Man's War", I stopped by the author's blog to see if he's mentioned the availability of his work through a particular net retailer. I discovered a pleasant surprise: The author is apparently well aware of Baen's success with un-DRM'd books, and his publisher, Tor appears to be willing to follow Baen's trailbreaking lead.

In this post, Mr. Scalzi announces that his book will be offered as a free download, as part of a promotion of some sort by Tor. What kind of a promotion? Well, we don't really know (and he didn't say), but most people have guessed that Tor is either going to open their own ebook store ala Webscriptions or extend their current relationship with Baen, based on the fact that titles, previously unavailable in any electronic format, are being offered for free and without the stigma of DRM.

Needless to say, I am quite excited by this discovery

There is, of course, a catch. By going to Tor's website and signing up for their promotion, you are, according to the privacy statement, agreeing to being spamed by not just Macmillan (who owns Tor), but also any 3rd parties they chose to share your information with. The ebooks are not free; the payment is becoming a marketing target. You do have the option of opting-out of the 3rd party marketing, but not Macmillan's.

Quote from Tor's privacy policy:
"We may share the information you provide with companies that are related to Macmillan through common ownership. We also share information provided by our website visitors with service providers we have retained to perform services on our behalf. In addition, we may share information about you with non-affiliated third parties whose products or services may be of interest to you. These third parties may contact you about their products and services."

So, the question becomes: Are 12 ebooks (quality books, such as Scalzi's "Old Man's War", and Brandon Sanderson's "Mistborn"), worth becoming a marketing target for a behemoth like Macmillan?

My answer is "Yes, as long as the things that are being marketed to me are of interest." In other words ... I don't want spam, I'm not interested in Viagra, time shares, or warez software. But marketing for books of authors and genres in which I have expressed an interest would be just fine. So, naturally, I quickly created an email account to sign up for the promotional event, and forwarded the mailbox to my real account. If I begin to receive spam emails targeted to the address I gave Tor, I can easily kill the mailbox and forget it ever happened. If the marketing proves to be useful, then everybody wins! Hooray!

Tags:   ebooks     |    Perm Link:   Tor gets serious about ebooks



James Conner