database on SWiK

Elixir – Trac

Fri, 29 Aug 2008 16:57:12 -0700

Elixir is a declarative layer on top of the SQLAlchemy library. It is a fairly thin wrapper, which provides the ability to create simple Python classes that map directly to relational database tables (this pattern is often referred to as the Active Record design pattern), providing many of the benefits of traditional databases without losing the convenience of Python objects.

hibernate.org - Hibernate Shards

Fri, 29 Aug 2008 15:56:57 -0700

BibSonomy ::

Fri, 29 Aug 2008 14:59:15 -0700

Async I/O to fix Ruby's green threads: Creators of NeverBlock and Revactor interviewed

Fri, 29 Aug 2008 13:58:55 -0700

Ruby 1.9's Fibers and non-blocking I/O are getting more attention - we talked to Mohammad A. Ali of the NeverBlock (Async PostgreSQL and MySQL drivers using Ruby Fibers) project and Tony Arcieri of the Revactor (Erlang-style Actors) project.

Welcome to Hadoop!

Fri, 29 Aug 2008 13:57:25 -0700

JODB - The Free Object Database for Java

Fri, 29 Aug 2008 11:58:04 -0700

mojoPortal - Setting a Data Layer Reference in MonoDevelop

Fri, 29 Aug 2008 08:56:39 -0700

Tokyo Cabinet: a modern implementation of DBM

Fri, 29 Aug 2008 06:56:53 -0700

heise online - 28.08.08 - Open-Source-Stack für Datenbank-Cluster

Fri, 29 Aug 2008 03:56:53 -0700

Der Tungsten Database Scale-out Stack von Continuent bietet Mechanismen zur Replikation von Open-Source- und proprietären Datenbanken.

Using Hibernate with the Java Persistence API

Fri, 29 Aug 2008 02:56:39 -0700

DBSlayer - Trac

Fri, 29 Aug 2008 02:05:45 -0700

Jitterbit Open Source Integration - Business Integration Made Easy.

Fri, 29 Aug 2008 02:04:41 -0700

Open Source Native XML Database

Fri, 29 Aug 2008 02:04:41 -0700

hibernate.org - Java Persistence with Hibernate

Fri, 29 Aug 2008 02:00:45 -0700

Hibernate, like all other object/relational mapping tools, requires metadata that governs the transformation of data from one representation to the other (and vice versa). As an option, you can now use JDK 5.0 annotations for object/relational mapping with Hibernate 3.2. You can use annotations in addition to or as a replacement of XML mapping metadata.

Hibernate Annotations

Fri, 29 Aug 2008 02:00:45 -0700

Sedna XML database

Fri, 29 Aug 2008 00:58:15 -0700

XML database - Wikipedia, the free encyclopedia

Fri, 29 Aug 2008 00:58:15 -0700

DBViewer Plugin for Eclipse

Fri, 29 Aug 2008 00:58:02 -0700

12 things I bet you didn't know about Hibernate | Learning by Experience

Fri, 29 Aug 2008 00:55:16 -0700

Open Source Native XML Database

Thu, 28 Aug 2008 23:58:01 -0700

Community

Thu, 28 Aug 2008 23:56:28 -0700

liba52 - a free ATSC A/52 stream decoder

Thu, 28 Aug 2008 23:51:43 -0700

Build a PL/SQL Web Service

Thu, 28 Aug 2008 20:57:48 -0700

How to Generate Hibernate Pojo Classes from DB Tables - wikiHow

Thu, 28 Aug 2008 18:09:43 -0700

NeverBlock, MySQL and MySQLPlus

Thu, 28 Aug 2008 18:09:05 -0700

I have great news for MySQL users. A very nice side effect has emerged from the development of the NeverBlock support for MySQL. I am glad to announce the release of a new MySQL driver for Ruby applications. It builds on top of the original Ruby MySQL driver but it comes with two notable additions

CSDN技术中心 Hibernate、iBATIS 与 BLOB

Thu, 28 Aug 2008 18:08:06 -0700

iBATIS - Accessing Multiple Instances of Databases from Your Application | Programming Resources

Thu, 28 Aug 2008 18:08:06 -0700

Apache iBATIS iBATOR

Thu, 28 Aug 2008 18:08:05 -0700

PyWorks Conference Schedule Posted

Thu, 28 Aug 2008 18:05:40 -0700

Hi all,

The schedule for PyWorks has been posted! I’m really excited about three things:

1) there are some really cool talks that I’m looking forward to attending. There are a couple of sysadmin-related talks, AppEngine, TurboGears, Django, and an area I’ve been especially slow to move into: testing (I know, shame on me). There’s lots more so be sure to check it out.

2) the conference scheduling process is over

3) I get to meet a lot of people face-to-face that I’ve worked with in the past on Python Magazine developing articles, or interacted with on IRC, etc. One thing I like about conferences surrounding open source technologies is you get to thank people face-to-face for the sweat they poured into some of the tools I use regularly. Mark Ramm, Kevin Dangoor, Michael Foord, Brandon Rhodes, and a collection of Python Magazine authors will be speaking there, and other Python Magazine folks and generally familiar faces will be in attendance.

Enjoy!

For those still unaware, PyWorks will be held in Atlanta, Nov. 12-14, 2008. It’s sponsored by MTA, the publisher of Python Magazine, as well as php|architect. In fact, the php|works conference will be held simultaneously with PyWorks, and attendees of one are free to access talks in the other at will. There will also be a “center track” that will cover some more generic topics of interest to developers without regard to the language in use. Check it out!

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F08%2F27%2Fpyworks-conference-schedule-posted%2F'; addthis_title = 'PyWorks+Conference+Schedule+Posted'; addthis_pub = 'jonesy';

More Adventures in Amazon EC2 (and EBS)

Thu, 28 Aug 2008 18:05:40 -0700

Short Version: You can find a fantastic video here about bundling customized AMIs and registering them with Amazon so that you can launch as many instances of your new AMI as you want. The video is so good that I don’t bother writing out the steps to do the bundling (it would be pretty darn long). These are some notes about launching an AMI, customizing it, and mounting an EBS volume to it (the video linked above doesn’t cover EBS). Also, check out the ElasticFox tool which is a very good GUI for doing simple EC2 operations. Nice if you’re just getting started or doing some simple tests.

There are two ways you can go about creating a custom machine image (AMI) for use with Amazon EC2: You can create an image locally by dd’ing to a file, mounting it with “-o loop” creating a filesystem on it, and bootstrapping the whole thing yourself, or you can grab an existing AMI that will serve as a “good enough” base for you to make your customizations, then bundle the customized image.

I’ll be talking about the latter option, where you identify a “good enough” image, customize it for your needs, and save that as your AMI. Unless you’re doing some kind of highly specialized installation, or are a control freak, you shouldn’t really need to start from scratch. I was just building a test image, and wanted a CentOS 5.2 base installation.

Here’s the command you can use to browse the AMIs you have access to (they’re either public, or they’re yours):

$ ec2dim -a

If that command looks funny to you, it’s likely because you’re used to seeing the really long versions of the AWS commands. Amazon also provides shorter versions of the commands. No, really - have a look! The long version of this command is:

$ ec2-describe-images -a

Too long for my taste, but it’s nice to know it’s there.

So, rather than start from scratch, I grabbed a base image that was close enough for my needs, and customized it. It’s a 5.1 base image, pretty well stripped of things that I don’t need, and a few that I do, but that’s ok. I’d rather start with less than more.

So step one is to launch an instance of the AMI I’ve chosen to be my ‘base’. Simple enough to do:

$ ec2run ami-0459bc6d -k ec2-keypair

And that’s pretty much it. It takes a couple of minutes (literally) for the machine to actually become available. You can check to see if it’s still in “pending” state or if it’s available by running ‘ec2din’. Without arguments, that’ll show you the status of any instances you have pending or runnning. Once the instance is running, you’ll be able to glean the hostname from the information provided.

An important note at this point: Don’t confuse “image” with “instance”. For the OO types in the crowd, an “image” is an object. It does nothing by itself until you instantiate it and create an “instance” of that object. For sysadmins, the “image” is like a PXE boot image, which does nothing until you boot it, thereby creating an “instance”.

The reason I used “PXE” and “object” in the above is because of the implication it makes: you can launch as many instances of an object as you want from a single object definition. You can boot as many machines as you want from a single PXE boot image. Likewise, you can launch as many Amazon EC2 instances from an image as you want.

So, in the time it took you to read those last two paragraphs, your instance is probably running. I now grab the hostname for my instance, and ssh to it using my keypair:

$ ssh -i ec2-keypair root@<hostname>

Now that I’m in, I can customize the environment, and then “bundle” it, which will create a new AMI with all of my customizations. With the instance in question, I installed a LAMP stack, and a few other sundry tools I need to perform my testing. I also ran “yum -y upgrade” which will go off and upgrade the machine to CentOS 5.2.

One thing I want to do with this instance is test out the process for creating an EBS volume. The two pieces of information I need to do this are the size of the volume I want to create, and the “zone” I want to create it in. You can figure out which zone your instance is running in using ‘ec2din’ on your workstation (not in your instance). I took that information and created my image in the same zone using the ‘ec2addvol’ command. If you don’t have that command on your workstation, then you don’t have the latest version of the Amazon command line tools. Here’s the command I ran:

$ ec2addvol -z us-east-1b -s 25

To see how it went, run ‘ec2dvol’ by itself and it’ll show you the status of all of your volumes, as well as the unique name assigned to your volume, which you’ll need in order to attach the volume to your instance. To do the ‘attachment’, you need the name of the volume, the name of the instance (use ‘ec2din’), and you need to choose a device that you’ll tell your instance to mount. Here’s what I ran (on my workstation):

$ ec2attvol -d /dev/sdx -i i-xxxxxxxx -v vol-xxxxxxxx

Now you can go back to the shell on your instance, mount the device, create a file system, create a mount point, add it to fstab, and, as they say in the UK, “Bob’s yer uncle”. By the time I wrote this post, I had already shut down my instance, but here are the commands (caveat emptor: this is from memory):

# mkfs.ext3 /dev/sdx

# mkdir /vol

# mount /dev/sdx /vol

If that all works ok, you can add a line to /etc/fstab so that it’ll be mounted at boot time, but I haven’t yet figured out how to attach a volume to an instance at boot time. The mount doesn’t work if you don’t attach the volume to the instance first. You’ll get a “device doesn’t exist” error if you try it. Clues hereby solicited. I assume I could probably use ‘boto’ and some Python code to get this done, but doing the same with a shell script wrapper around the Amazon tools might also be possible — but I don’t know how reliable that would be, because you’re at the mercy of Amazon and how they decide their tools should present the data (and *if* they provide the data you need for a particular operation down the road).

So now I have an EBS volume, and an instance. The volume is attached to the instance, and I can do things with it. I’m testing some database stuff, so I copied a database over to the volume, which was now mounted, so I could just ’scp mydb.tbz root@<instance>:/vol/.’

Once my database is there, I can attach it to pretty much whatever I want, which makes it nice, because I can test the same database, and the same database code, and see how the different size Amazon instances affect the performance, which gives me more performance data to work with. For production purposes, I’ll have to look more closely at the IO metrics, play with attaching multiple volumes and spreading out the IO, and I also want to test the ’snapshot’ capabilities. It’s also nice to know that if I needed to launch this in production (there are no plans to do so, but you never know), I could upgrade the database “hardware” more or less instantly

If anyone has code or tools to help automate the management of all of this stuff, please send links! If I come up with any myself, I’ll most likely post it here.

Now that I have a customized AMI with all of my packages installed and my config changes made, I need to bundle this so that I can boot as many instances of this particular configuration as I want. An important note about bundling this *particular* image is that you MUST run ‘depmod -a; modprobe loop’ before bundling, since this process basically abstracts the manual process of bundling an image, which involves mounting a file as a volume, which requires a loopback mount.

The video I used to do the bundling is here, and if you can live through the disgustingly bad burps and chirps in the (Flash version) audio, it’s an excellent tutorial for bundling custom AMIs. While the process *is* pretty straightforward, it involves a number of steps, and the video goes through all of them, and it worked perfectly the first time through.

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F08%2F27%2Fmore-adventures-in-amazon-ec2-and-ebs%2F'; addthis_title = 'More+Adventures+in+Amazon+EC2+%28and+EBS%29'; addthis_pub = 'jonesy';