BeOS Device Driver Development Hints and Information

This document attempts to describe how to develop device drivers for the BeOS (and BeIA) operating systems. It is also useful to the developer of an application which has to talk to a device driver. In order to set the stage for the driver development discussion, I will first discuss how to use a device driver from an application.

Using Device Drivers on BeOS

Typically, a user-level application will not use a BeOS device driver directly, instead using some higher-level API which calls a user-level add-on which calls the driver. However, there is nothing that prevents an application from talking directly to a driver just like an appropriate add-on would; indeed, in some cases there is no appropritate add-on API and you have to talk to the driver directly from an application. It is also useful to talk directly to the driver while developing and testing the driver. We will call the entity (application or add-on) using the driver a "client" of the driver.

First, finding the device. The driver will export one or more devices in sub-directories of the "/dev" directory. For instance, the sonic_vibes audio card driver export in /dev/audio/raw/sonic_vibes/ as well as in other locations. Because most BeOS drivers support handling more than one installed card of the same kind, the convention is to number the installed cards, starting at 1. Thus, the first installed sonic_vibes card is found as /dev/audio/raw/sonic_vibes/1. You can use the BDirectory class or the opendir() C function to look through a directory for available devices.

Once you know what device you want to use, you should open it using the open() C call:

	int fd = open("/dev/audio/raw/sonic_vibes/1", O_RDWR);

You will use this file descriptor to refer to the open device from now on. The file descriptor should be closed with close() when you are done with it. If the process (team) that opened the device crashes or otherwise goes away without closing the file descriptor, it will be garbage collected and closed by the kernel.

Many devices implement the read() and write() protocols. Thus, to record some audio from the default input source, using the raw audio file descriptor opened above, all you have to do is:

	short * data = (short *)malloc(200000);
ssize_t rd = read(fd, data, 200000);

"rd" will contain the number of bytes actually read, or -1 if an error occured (in which case the thread-local macro "errno" will contain the error code).

The format of the data returned by the device varies with what device it is; the default format of the sonic_vibes driver is stereo 16-bit signed native-endian 44.1 kHz PCM sample data. To play back this data using the write() call, do this:

	ssize_t wr = write(fd, data, rd);

"wr" will contain the actual number of bytes written, or -1 for error, in which case "errno" contains the error code.

Many devices do not work well with the simple read() and write() protocol; for instance, video capture cards often require a contiguous locked area of memory, which typically is not found in a buffer passed in by the user to read() or write(). Then you can implement your protocol as ioctl() selectors. There are a number of well-defined ioctl() values that your device can implement if they make sense for the class of device you're dealing with; specific sub-directories of /dev may require certain ioctl() protocols to be implemented (such as /dev/joystick, /dev/midi, or /dev/audio).

Supposing we're using a video capture driver which implements the following protocol:

	enum {
drvOpSetBuffers = B_DEVICE_OP_CODES_END+10001,
drvOpStart,
drvOpStop,
drvOpWaitForFrame,
};
struct drv_buffer_info {
color_space in_space;
int in_width;
int in_height;
int in_rowbytes;
void * in_buffers[2]; /* even, odd */
};
struct drv_frame_info {
int out_frame_number;
};

The client could then configure the driver like so:

	drv_buffer_info buf_info;
buf_info.in_space = B_YUV422;
buf_info.in_width = 640;
buf_info.in_height = 240;
buf_info.in_rowbytes = 640*2; /* 422 is 16bpp */
area_id buf_area = create_area("capture buffers", &buf_info.in_buffers[0],
B_ANY_ADDRESS, buf_info.in_rowbytes*buf_info.in_height*2,
B_CONTIGUOUS, B_READ_AREA|B_WRITE_AREA);
buf_info.in_buffers[1] = ((char *)buf_info.in_buffers[0])+buf_info.in_rowbytes*buf_info.in_height;
int err = ioctl(fd, drvOpSetBuffers, &buf_info);
if (err == -1) err = errno;

It would start video capture like so:

	int err = ioctl(fd, drvOpStart);
if (err == -1) err = errno;

It would wait for each frame to arrive like so:

	while (running) {
drv_frame_info frm_info;
int err = ioctl(fd, drvOpWaitForFrame, &frm_info);
if (err == -1) err = errno;
process_frame(frm_info.out_frame_number, buf_info.in_buffers[frm_info.out_frame_number & 1]);
}

Last, it would stop the capture like so:

	int err = ioctl(fd, drvOpStop);
if (err == -1) err = errno;

In real life, a typical protocol will be more capable, and thus more complicated, than shown here, but it should be enough to give you an idea of how the protocol between a user-level client and a driver can be structured. Also, in real life, drivers know better about their buffering requirements than applications, so the API would probably have an ioctl() to allocate buffer(s) and return information about them to the client, rather than have the client tell the driver which buffer(s) to use.

Writing Device Drivers on BeOS

OK, now that you know how to use a device driver, and have some idea of how to structure the protocol between the client and the driver, it's time to get down and dirty with the actual process of creating a driver. Creating a driver on BeOS is done using ANSI C; the C++ language requires certain support which is not available in the BeOS kernel environment. If you already have a large C++ library that talks to your hardware device and want to port it to BeOS, we suggest that you make your driver very shallow and use it just to read/write card registers and service interrupts, and put all your C++ code in a user-level add-on. Some readers may know "interrupts" by the name "IRQ"; we will call them "interrupts" because that's the terminology used by the BeOS kernel kit.

A driver is a loadable shared library (add-on) which exports certain well-known function names such as init_driver() and publish_devices(). The driver gets loaded by the "devfs" file system (which runs in the kernel) in response to some client calling file system functions opendir(), open(), and others. A driver may get loaded and unloaded several times, not necessarily being opened just because it's loaded. It will, however, never be unloaded while it is open. The moral of this story is that you cannot expect global or static variables to retain their values after uninit_driver() has been called, or before init_driver() is called.

Drivers, Devices and Hardware

First, you have to decide on what to call your driver and your devices. Typically, one driver will service any number of installed cards of the same kind, and each of those cards may cause the driver to publish multiple device names under /dev. These device names will be referred to as "devices"; the actual binary add-on will be called the "driver" and the pieces of hardware serviced by the driver will be called the "hardware".

Typically, you will name your driver something close to the name of the main chip serviced by the driver. The sonic_vibes driver drives the S3 Sonic Vibes chip; the bt848 driver drives the Brooktree Bt848/878 chips; the awe64 driver drives the Creative Labs SoundBlaster AWE32/64 cards; etc.

Your device names will then be derived from the protocols they implent, as well as the driver name. Thus, sonic_vibes publishes devices in /dev/audio/raw/sonic_vibes, /dev/audio/old/sonic_vibes, /dev/audio/mix/sonic_vibes, /dev/audio/mux/sonic_vibes, /dev/midi/sonic_vibes and /dev/joystick/sonic_vibes, each device implementing the protocol that's defined for that part of the /dev directory tree. If there is no protocol defined for your kind of device, you can implement whatever protocol you wish. Try to stay consistent in your naming, however. For instance, a video capture driver for a chip named Pixtor might publish devices in /dev/video/pixtor/. By convention, each card will be numbered from 1 and up, so the first "pixtor" device would be called /dev/video/pixtor/1.

If your device is of some irregular kind, you can always publish in /dev/misc/your-name. Please avoid publishing directly under /dev, and avoid inventing new classes of devices under /dev/. If you feel you have to, please first contact BeOS developer support or your favourite Be engineer to check that your scheme will work well with the rest of the system.

Life and Death

The first function called in your driver, if you implement and export it, is the init_hardware() hook. Please refer to the skeleton driver for the C function prototype of each driver function.

init_hardware() will only be called the first time your driver is loaded, to find and reset your hardware and get it into some known state, if necessary. Many drivers can do without implementing this hook at all. If you implement this hook, but don't find any of your cards installed, you should return a negative error code, such as ENODEV. Note: on BeOS, all the POSIX error codes (EXXX) are negative numbers, so you should return them as-is to signify error.

The next driver hook being called is init_driver(), which definitely should be implemented by all drivers. If your device is an ISA card, you will want to call get_module() on the ISA bus manager module to initialize a global variable to refer to that module for easy access (typically, this variable will be named "isa"). For PCI cards, use the PCI bus manager module, found in <PCI.h>.

Then, use the bus manager module to iterate over available hardware, looking for instances of the hardware you support. For each piece of hardware, make sure you enable its PCI bus interface in the configuration registers if it isn't already. Then allocate whatever memory you need to keep track of the hardware and the devices that hardware will cause to be published, and make sure the hardware is in some safe, well-behaved state and not generating spurious interrupts or other bad behaviour. To allocate memory, use malloc(). To later deallocate this memory, use free().

	/* a global variable for the PCI module */
pci_module_info * pci;

/* in init_driver() */
pci_info info;
int ix = 0;
int cards_found = 0;
if (get_module(B_PCI_MODULE_NAME, (module_info **)&pci) < 0)
return ENOSYS;
while ((*pci->get_nth_pci_info)(ix, &info) == B_OK) {
if (info.vendor_id == MY_VENDOR && info.device_id == MY_DEVICE) {
cards_found++;
my_card_array[ix].info = info;
}
ix++;
if (cards_found == MAX_CARDS) break;
}
if (cards_found < 1) return ENODEV;
names[cards_found] = NULL;

/* in uninit_driver() */
put_module(B_PCI_MODULE_NAME);

If you find no hardware, return ENODEV. If you find hardware, but something is wrong and you're not prepared to publish any devices, return ENOSYS or ENOENT. If all is OK, return B_OK.

Next, the hook publish_devices() will be called. It should return a pointer to an array of C string pointers, one per device you want to publish, and terminated by a NULL pointer. For a hypothetical "Pixtor" driver which publishes one device per installed hardware card, up to a maximum of four installed cards, you will typically have a global variable "names" like so:

	static char * names[5] = {
"video/pixtor/1",
"video/pixtor/2",
"video/pixtor/3",
"video/pixtor/4",
NULL /* init_driver() sets unavailable slots to NULL */
};

In init_driver() you will allocate a name string per device you find (unless a static array will work, as shown), and make the corresponding slot in "names" point to that string. Then you can just return the "names" array in publish_devices():

	const char ** publish_devices() {
if (names[0] == NULL) return NULL;
return names;
}

Note that the names assume to live under "/dev/" and thus should NOT contain that part; a typical name may be "video/pixtor/1".

Naming Detour

How does the devfs file system know which driver to open when a program asks for the device named "/dev/foo/bar/1"? Under R3, devfs opened all drivers when the system booted and called their publish_devices() function, so it could know what devices were available. However, this mechanism doesn't scale well with an increasing number of drivers available for BeOS, and a new mechanism was introduced in R4.

Inside /system/add-ons/kernel/drivers (and ~/config/add-ons/kernel/drivers) there are now two folders, "dev" and "bin". All driver binaries go into "bin", and symlinks to the drivers go into the appropriate subdirectory of "dev". Thus the hypothetical Pixtor driver would put the driver in ...kernel/drivers/bin, and put a symlink to that driver in ...kernel/drivers/dev/video. The symlink has to be put there by the installation program or script for the driver, or, for development purposes, by the driver build process.

Thus, when a client calls open("/dev/video/pixtor/1") or opendir("/dev/video/"), devfs will scan all symlinks found in ...kernel/drivers/dev/video (and subdirectories thereof) and open the referenced drivers to call their init_driver() and publish_devices() functions, in order to figure out which driver(s) publish devices that would interest the client. Devfs is reasonably smart about only doing this once, and it uses the modification date of the driver in .../bin to do that, so when you replace your driver with a newer copy, subsequent open() calls for your driver will cause devfs to load the new version (once all the old clients have closed the old driver). Not having to reboot for the new driver to be found is one of my favourite features of BeOS for driver development.

Note: as of Release 4.5, the kernel may be even smarter about not loading a driver again, so your make file can use the rescanshell command to tell devfs that your driver is updated. Give it the name of your driver binary (found in ...kernel/drivers/bin) like so:

rescan sonic_vibes

Getting to Open

When a client decides to open one of your devices, the kernel will call your find_device() hook with the name in question. It is up to you to map this name (which you previously published in publish_devices()) to the right device type within your driver. If you support only one device type, this is easy, and even if you support more than one, a simple strncmp() will typically be sufficient.

A "device type" consists of a set of function pointers that define the interface for a device. In <Drivers.h> you will find the struct device_hooks which is what should be returned from find_device(). The hooks for open, close, free, read, write and control are required to be implemented; the hooks for select, deselect, readv and writev are optional. If you don't implement select/deselect, the kernel will return an error to the client of your file descriptor if the hook is used. If you don't implement readv/writev, the kernel will emulate these functions by repeatedly calling your read/write hooks, which may be less efficient than if you supported the readv/writev functions directly. Don't confuse the hook name "select()" with the net kit function "select()"; currently they have nothing to do with each other.

Once you have returned a device_hooks structure, the kernel will call the open hook therein, letting you turn the device name into a unique "cookie" which your other hooks will use to find the open device in other hook calls. The open mode is O_RDONLY, O_WRONLY, or O_RDWR. Depending on your driver's capabilities, you might want to ensure exclusive access to reading and writing respectively, and return a EPERM error if someone tries to open the same device with the same mode twice in a row. It may, however, make sense to allow one open() for O_RDONLY and another open() for O_WRONLY.

The kernel will never dereference the "cookie" value, so it can be a pointer to some private data you malloc(), or a pointer to some element in a global array, or just an index of some sort. Suffice to say that you must be able to get all necessary state information for the open device, and the hardware associated therewith, when given this cookie in later hook function calls.

Your open() hook will typically need to call install_io_interrupt_handler() to install an interrupt service routine for the hardware in question the first time it is opened, if you didn't already do that in init_driver(). For PCI devices, you find the values to pass to this function in the pci_info struct for your hardware. The "data" value will be passed to your interrupt handler, and thus is typically your "cookie" value.

Note that the device_hooks structure may acquire more functions in later versions of BeOS. To tell the kernel what version of the interface you were compiled with, you should export an int32 variable named api_version, it should be initialized to B_CUR_DRIVER_API_VERSION. Assuming you put your device_hooks structs in static or global memory, the compiler will clear out any slots you don't define at the end to NULL for the version of the device_hooks struct you compile with; thus the value of B_CUR_DRIVER_API_VERSION will change when the size of the device_hooks struct changes. Just adding this line to your driver is enough, as long as you include <Drivers.h> before it:

	int32 api_version = B_CUR_DRIVER_API_VERSION;

Close and Free

When the user is done with your device, he will call close() on the file descriptor that references it. When the file descriptor is closed (or, when the last file descriptor is closed, if the user uses dup()), the kernel will call your close() hook. You should start shutting down the device; set a status bit so that future read(), write() and control() hook calls will return an error, and preferrably un-wedge any outstanding blocking I/O requests and have them return EINTR. One technique of doing this is to simply delete the semaphores you use for synchronizing I/O. The acquire_sem() calls in your driver hooks should then detect the B_BAD_SEM_ID error and take that to mean that the device is being shut down, and return EINTR to the calling client.

Once all outstanding I/O requests have returned out of your driver, the free() hook will be called. Here is where you can deallocate all memory you allocated in open() or during the course of dealing with the specific open device (as indicated by the cookie), and re-set your driver to accept a future open() for that device name. Note that there will be exactly one call to the free() hook for each call to the open() hook, and that call to free() for the cookie returned by open() will always come after a call to close() for that cookie. There is no relation between different cookies returned by different calls to open(), as far as the kernel knows they are independent.

The free() hook is a good place to call remove_io_interrupt_handler() to remove the interrupt handler for your device if you installed it in open(). If you allow multiple open()s, it's easier to install the handler (once) in init_driver() and remove it in uninit_driver(); don't install a handler more than once for the same hardware! Pass the same "data" value as you passed to install_io_interrupt_handler() in open() (i e, for most devices, your "cookie" value).

Your Interrupt Handler

Your interrupt handler will be called whenever an interrupt on your interrupt number occurs. Because of interrupt sharing, your hardware may not be the hardware that generated the interrupt. Your interrupt handler will be called on to figure out whether the interrupt was caused by your hardware, and if so, to handle it. You should read the appropriate status register on your hardware the first thing you do in the interrupt handler, and if the interrupt was not generated by your hardware, immediately return B_UNHANDLED_INTERRUPT. This will let the kernel move on to other interrupt handlers installed for the same interrupt number and see if they can handle the interrupt.

If the interrupt was indeed generated by your hardware, you can go ahead and handle the interrupt, and then return B_HANDLED_INTERRUPT.

While your interrupt handler runs, interrupts are turned off. Thus, threads cannot be re-scheduled, and other interrupts cannot be handled. This means that your interrupt handler should run as fast as possible. A typical interrupt handler just acquires a spinlock (for mutual exclusion with user-level threads), adjusts some internal data structure, and quite possibly releases a semaphore which the user thread (read(), write() or control()) is waiting on. Because re-scheduling with interrupts disabled can cause a total system hang, you should release semaphores using release_sem_etc() and pass the B_DO_NOT_RESCHEDULE flag like so:

	release_sem_etc(my_cookie->some_semaphore, 1, B_DO_NOT_RESCHEDULE);

The scheduling quantum on BeOS is 3000 microseconds. Thus, if you release a semaphore without re-scheduling, the longest you may have to wait before a re-schedule happens, and the schduler gets a chance to notice that your semaphore has become available, and thus be able to schedule the thread waiting on the semaphore, is 3 milliseconds. If this is too long time (for low-latency media devices like audio and MIDI, for example) your interrupt handler routine can return the special value B_INVOKE_SCHEDULER, which means that you handled the interrupt, and want a thread reschedule to happen at the earliest possible time. The kernel will then call resched() as soon as it leaves interrupt level, which will give the scheduler a chance for noticing that your semaphore has been released and your waiting thread is now ready to run.

Note that, because of multi-threading and thread priorities, your thread may not be the thread chosen to run just because a reschedule happens. If you have really low latency requirements, and can't afford to have lower-priority threads come between your interrupt handler and your waiting thread getting scheduled, you have to use real-time priority for the thread waiting for the interrupt. Using real-time priority for threads is dangerous, however, because they may completely lock out other threads from the system, including the graphics threads that draw to the screen, making the system appear "hung" if your real-time thread does too much work without synchronizing with a blocking primitive (like a semaphore).

Read(), Write() and Control()

Now that you know how your device gets loaded and unloaded, and how to handle interrupts generated by your hardware, you can design the rest of your device API to be used by user-level clients.

The read() hook gets called in repsonse to a call to the user-level function read() on a file descriptor that references your device. The cookie for your device will be passed to the read() hook, as well as the current position, as maintained by the kernel file descriptor layer. If your device does not support positioning (seeking) you can ignore the position parameter.

Your job inside the read() hook is to transfer data into the buffer passed into the read() hook. The buffer has a size of *numBytes. You should transfer at most that many bytes, and then set *numBytes to the number of bytes transferred. If any bytes were transferred, return B_OK. If some error occured and/or no data was transferred, set *numBytes to 0 and return a negative error code.

Please note that the buffer pointed at by "data" will typically be in the user space of the team calling read(). It will typically be in discontiguous memory, and it will not be locked in physical RAM. Thus, it is not accessible from an interrupt service routine, nor can you DMA directly into it without first locking the buffer and getting the physical memory mapping for it:

	status_t
read_hook(void *cookie, off_t position, void *buffer, size_t *numBytes)
{
status_t err;
long entries = 2+*numBytes/B_PAGE_SIZE;
physical_entry * pe = (physical_entry *)
malloc(sizeof(physical_entry)*entries);
status_t err;
my_device * md = (my_device *)cookie;

err = acquire_sem_etc(md->lock_sem, 1, B_CAN_INTERRUPT);
if (err < 0) {
/* delete_sem() from close(), or signal interrupting */
free(pe);
return err;
}
err = lock_memory(buffer, *numBytes, B_DMA_IO);
if (err < B_OK) {
free(pe);
release_sem_etc(md->lock_sem, 1, B_DO_NOT_RESCHEDULE);
return err;
}
entries = get_memory_map(buffer, *numBytes, pe, entries);
/* set up and start your DMA here */
...
/* assume your interrupt handler will release this sempahore when DMA done */
err = acquire_sem(md->my_dma_sempahore);
unlock_memory(buffer, *numBytes, B_DMA_IO);
if (err < B_OK) {
*numBytes = 0;
}
free(pe);
/* use B_DO_NOT_RESCHEDULE because we want to return to caller */
/* with data without further delay */
release_sem_etc(md->lock_sem, 1, B_DO_NOT_RESCHEDULE);
return err;
}

The same rules apply for the write() hook, except of course the data transfer direction is from the buffer passed by the client to your hardware.

An alternative is to use a contiguous buffer in kernel space that you allocate and copy to/from in read() and write(). If you have a sound card that uses a cyclic auto-repeat DMA buffer, this is often a good solution, for example. However, if the data rate is high, such as for live video or fast mass storage devices, you want to avoid copies. You might choose to just have read() and write() return an error, and use ioctl() exclusively for communicating with your device. Another option is to make ioctl() the preferred protocol, but have read() and write() call the appropriate ioctl() functions for convenience.

These kinds of decisions are easier if you're implementing a device for which Be has defined a protocol, because then you just follow the protocol. However, if your're implementing a driver for a device for which there is no pre-defined protocol, or if your device will have significantly better performance using some other protocol, you will have to design the driver protocol on your own.

The control() hook is called in response to the user level client calling the ioctl() function:

	struct the_args {
int a;
int * b;
};
int foo;
struct the_args args;
args.a = 1;
args.b = &foo;
err = ioctl(fd, SOME_CONSTANT, &args);
if (err == -1) err = errno;

The control() hook will receive the integer constant passed to ioctl(), as well as the pointer argument. Currently, the "size" argument will always be 0 when passed to the hook, so you can ignore it. Assume that the pointer argument is correct for the integer constant in question.

You can start numbering your own operation constants from B_DEVICE_OP_CODES_END+1 (in <Drivers.h>). If you want to avoid the risk of clashing with someone trying to use a protocol you do not know about on your device, you can choose some arbitrary, larger number to start numbering from, such as your birthday or something. As long as the numbers (when read as signed 32-bit integers) are larger than B_DEVICE_OP_CODES_END.

In the example above, your device control hook can look like this:

	status_t
control_hook(void * cookie, uint32 operation, void * data, size_t length)
{
my_device * md = (my_device *)cookie;
status_t err = B_OK;

switch (operation) {
case SOME_CONSTANT: {
struct the_args * ta = (struct the_args *)data;
int i;
if (ta->a > MAX_INDEX_FOR_MY_DEVICE) {
ta->a = MAX_INDEX_FOR_MY_DEVICE;
}
if (ta->b == NULL) {
err = B_BAD_VALUE;
}
else {
err = acquire_sem(md->lock_sem);
if (err < B_OK) {
return err;
}
for (i=0; i<ta->a; i++) {
ta->b[i] = md->some_value[i];
}
release_sem(md->lock_sem);
}
}
break;
default:
err = B_DEV_INVALID_IOCTL;
break;
}
return err;
}

Some Rules to Follow

Semaphores may cause a re-schedule to another thread when released. Thus, you should not release a semaphore from an interrupt handler, or with interrupts disabled, without passing the B_DO_NOT_RESCHEDULE flag (using release_sem_etc()).

It is generally a good idea to put as much code as possible at the user level, and make your driver as shallow as possible even if you aren't forced to by porting C++ code. The less code there is in the driver, the less locked memory will be used, and the less code there is that may crash the kernel. All of your driver's code and global/static data, as well as all memory returned by malloc() called from a driver, is locked (and thus safe to access from an interrupt handler). Be gentle on the system.

Disabling interrupts is NOT sufficient to guarantee atomicity, because on a SMP system, the other CPU may be calling into your driver at the same time. For synchronization with data accessed by interrupt handlers, you have to use a spinlock. Spinlocks are the most primitive synchronization mechanism available; basically they use some atomic memory operation to test-and-set a variable. When the test-and-set fails, the calling thread just keeps trying (busy-waiting) until it succeeds. Thus, contention for spinlocks can be quite CPU intensive. Therefore, they should be used sparingly, and only to synchronize data that really has to be touched by an interrupt handler (since semaphores cannot be used by interrupt handlers).

A spinlock is simply a int32 value in some permanent storage (a global, or some memory you malloc() as part of opening your device) that is initialized to 0 before being used the first time. To acquire a spinlock, you turn off interrupts and then call acquire_spinlock():

	/* These are global variables. */
int32 my_spinlock = 0;
char protected_data[128];
int protected_ctr = 0;
	/* Acqurie spinlock. */
cpu_status cp = disable_interrupts();
acquire_spinlock(&my_spinlock);
/* Do protected operations -- this should be fast and not cause */
/* any reschedule, so don't call malloc() or any semaphore operations */
/* or any function that may call these functions. */
protected_data[protected_ctr++] = 0;
protected_ctr = protected_ctr & 127;
/* Release spinlock. */
release_spinlock(&my_spinlock);
restore_interrupts(cp);
	/* in your interrupt handler */
/* serialize with user code, possibly on other CPUs */
acquire_spinlock(&my_spinlock);
/* Do protected operations like hardware register access */
release_spinlock(&my_spinlock);

If you fail to disable interrupts before acquiring the spinlock, you will deadlock on single-CPU machines, because your interrupt handler may then be called (and try to acquire_spinlock() your spinlock) while the regular thread is holding the spinlock. That would be bad.

Many people find it convenient to wrap spin-locking into two general-purpose lock() and unlock() routined to not forget to turn off interrupts. You can use the same routines inside your interrupt handler, because calling disable_interrupts() and later restore_interrupts() is OK even inside an interrupt handler (even though interrupt handlers run with interrupts already turned off). Spinlocks, like sempahores, don't nest like that, however, so think about what you're doing and don't call functions that may lock a spinlock from some function that already holds the same spinlock.

	/* assuming you keep state information about your hardware in a */
/* struct named my_card, with a 0-initialized spinlock named hardware_lock */

cpu_status lock_hardware(my_card * card)
{
cpu_status ret = disable_interrupts();
acquire_spinlock(&card-<hardware_lock);
return ret;
}

void unlock_hardware(my_card * card, cpu_status previous)
{
release_spinlock(&card-<hardware_lock);
restore_interrupts(previous);
}

It is important to not disable interrupts for a long time. As a general rule, no more than 50 microseconds is allowable. If you disable interrupts for longer, you will jeopardize the overall performance of the BeOS and the machine it is running on. Similarly, your interrupt service routine should not run for more than 50 microseconds (and less is, of course, better).

You may find the lack of deferred procedure calls disturbing if you come from some other driver architecture. However, the time it takes for BeOS to service an interrupt, release a semaphore, and cause a re-schedule into a user-level real-time thread is often smaller than the time it takes for other operating systems just to handle the interrupt and get to the deferred procedure call level. Thus, we prefer to do what needs to be done in the user level client threads that call into the device hooks.

Don't Use Kernel Threads in Drivers

If at all possible, let the user of your device spawn whatever threads your device needs. Kernel threads are very tricky and live by different, undocumented rules. If you find a need for a periodic task, you can look into using timer interrupts (available as of BeOS R4.1). Look for add_timer() and cancel_timer() in <KernelExport.h>.

If you find a wish to use a kernel thread in your driver, there are several pitfalls that makes doing so a bad idea. The kernel team is a team just like any other team, and your kernel thread will have a stack in the upper half of the kernel team address space. This stack (and stacks of other kernel threads) is not accessible from user programs, and thus it is not accessible from device hooks called by user programs. Only the lower half of the kernel team address space (0x0-0x7fffffff) is accessible to all teams when they enter the kernel. This also implies that the stack of a kernel thread, or any other thread, is not accessible from an interrupt handler, since the MMU context of an interrupt handler is undefined; only the kernel space 0x0-0x7fffffff is accessible from an interrupt handler. Driver binaries and kernel malloc() memory live in this accessible space.

You also cannot use wait_for_thread() in your close() or free() hooks, because doing so will cause a deadlock with the psycho_killer thread which is responsible both for reaping dead threads and for freeing file descriptors and their associated devices. Thus, it is impossible for you to make perfectly sure that your kernel thread has terminated before your free() hook returns. This is a problem big enough that you should reconsider using kernel threads at all in your driver, if there is some other possibility. (This specific problem has been fixed in Release 4.5 of the BeOS, but all the other problems with kernel threads will still remain) Release 4.0 will be the baseline BeOS for some time to come, so a design without kernel threads is thus more widely compatible.

Again: Put the threads you need in the user-level client (add-on, application, whatever). If the driver needs to perform periodic tasks not in response to hardware interrupts, use timer interrupts. They are even more lightweight than threads, and have fewer of the problems mentioned above. As with any interrupt routine, timer interrupts still cannot access memory that is not in the kernel space; thus if you need to write into user-supplied buffers, do it in your driver read(), write() and control() hooks.

Good luck!

B500 logoNetPositive logo
buy hydrocodone effects of hydrocodone hydrocodone addiction buy hydrocodone online buy hydrocodone without prescription buy hydrocodone without a prescription buy hydrocodone online without a prescription buy hydrocodone no prescription buy hydrocodone online without prescription buy hydrocodone no rx buy hydrocodone without perscription easy way to buy hydrocodone online buy hydrocodone cod buy hydrocodone illegally medicines hydrocodone buy lortab online how to buy hydrocodone buy hydrocodone from mexico without a prescription how to buy hydrocodone without prescription buy hydrocodone online no prescription buy hydrocodone without prescription online consultation where can i buy hydrocodone online online pharamacy to buy hydrocodone buy hydrocodone with free consult buy hydrocodone online from online pharmacies buy hydrocodone online without precription buy hydrocodone cash on delivery from online pharmacies buy cheap hydrocodone no scrip buy hydrocodone without prescription overseas buy hydrocodone with echeck forum discussion how to buy hydrocodone online hydrocodone lorcet buy buy hydrocodone only by cash on delivery hydrocodone buy no prescription buy hydrocodone safely buy hydrocodone mexico laws buy hydrocodone h1 buy hydrocodone h1 a the now td buy hydrocodone money orders buy hydrocodone online in nevada buy hydrocodone online no prescription no medical records buy hydrocodone without prior prescription buy hydrocodone with no prescription buy lorcet and hydrocodone online cp hydrocodone buy viagra buy hydrocodone vicodin purchase pain medication online erowid hydrocodone buy tramadol now buy lortab hydrocodone buy hydrocodone presription buy hydrocodone prescription tramadol buy hydrocodone online2c forum extraction hydrocodone buy now tramadol buy hydrocodone robots txt buy hydrocodone tramadol free online consultation buy hydrocodone vicodin fedex hydrocodone line buy phentermine nt buy hydrocodone hydrocodone test urine buy soma online hydrocodone buy vicodin no prescription vicodin hydrocodone buy vicodin where can i buy hydrocodone hydrocodone online buy pain medication home hydrocodone online buy hydrocodone buy pain meds percocets how to buy hydrocodone online buy hydrocodone online vicodin hydrocodone liquid buy phentermine hydrocodone m360 buy tramadol guaifenesin hydrocodone syreth buy tramadol hydrocodone symptom withdrawal buy tramadol buy hydrocodone 26 codeine rx buy hydrocodone 10325 without a prescription buy hydrocodone cheap buy hydrocodone cheap without prescription buy hydrocodone consultation buy hydrocodone club join buy domain hydrocodone pisem net buy cheap nevada based hydrocodone 24 buy hydrocodone buy hydrocodone online from online pharmacy best websight to buy hydrocodone buy cheap hydrocodone no prescription buy cheap hydrocodone online buy hydrocodone easy buy counter hydrocodone buy hydrocodone no perscription buy hydrocodone no consultation buy hydrocodone no prescription online buy hydrocodone elixir buy hydrocodone on the internet buy hydrocodone no record buy hydrocodone lortabxanax buy hydrocodone no medical records buy hydrocodone eu agrees turkey cyprus response buy hydrocodone ibuprofen buy hydrocodone from mexico buy hydrocodone in lots of 100 pills or more buy hydrocodone legitimate place buy hydrocodone line from hydrocodone withdrawal buy viagra extraction hydrocodone buy viagra from hydrocodone withdrawal buy tramadol now extraction hydrocodone buy phentermine erowid hydrocodone buy now tramadol effects hydrocodone side buy tramadol now guaifenesin hydrocodone syreth buy fioricet effects hydrocodone side buy viagra effects hydrocodone side buy tramadol hydrocodone buy on line hydrocodone buy online hydrocodone buy us doctors secure site fedex effects hydrocodone side buy now tramadol hydrocodone buy without prescription hydrocodone buy hydrocodone and buy and credit card and no consultation homatropine hydrocodone buy tramadol now how to buy hydrocodone in mexico hydrocodone and buy guaifenesin hydrocodone syreth buy phentermine buy vicodin hydrocodone buy over the counter hydrocodone buy phentermine and hydrocodone com hydrocodone cheap buy no consultations buy vicodin hydrocodone lortab fedex buy now hydrocodone without a consultation or pr buy no rx hydrocodone online buy hydrocodone without prior perscription buy mass quantities hydrocodone buy no prescription hydrocodone online can i buy pain medicine online hydrocodone can you buy hydrocodone in cancun contact lenses buy hydrocodone digital printing dvd rw domain buy hydrocodone tripod com drug hydrocodone testing buy now tramadol drug hydrocodone testing buy tramadol now cods hydrocodone buy now tramadol cheap hydrocodone buy cheap hydrocodone online cash delivery hydrocodone buy fioricet cash delivery hydrocodone buy now tramadol cash hydrocodone buy now tramadol drugs online buy hydrocodone no prescription hydrocodone to buy online prescription hydrocodone and buy oxycontin hydrocodone vicodin buy no prescription pain relief buy hydrocodone prescription buy hydrocodone online pharamacy to buy hydrocodone without perscription no prescription order buy hydrocodone online lortab buy hydrocodone lortab norco buy hydrocodone vicodin pha massachusetts buy hydrocodone now sites to buy prescription hydrocodone vicodin buy hydrocodone where can i buy vicodin or hydrocodone online where to buy hydrocodone ibuprofen where to buy over the counter hydrocodone buy hydrocodone without prescription drugs canada where can i buy hydrocodone without a perscription where can i buy hydrocodone cash on delivery vicodin hydrocodone buy xanax order valium online alprazolam websites to buy hydrocodone without written prescription where can i buy hydrocodone lethal dose ambien valium hydrocodone andnot buy online legitimate places to buy hydrocodone hydrocodone mexican pharmacy buy tramadol now hydrocodone online buy tramadol now hydrocodone online buy vicodin without a prescription hydrocodone online rx buy now tramadol hydrocodone mexican pharmacy buy fioricet hydrocodone liquid buy viagra hydrocodone com buy hydrocodone line buy fioricet hydrocodone liquid buy tramadol hydrocodone prescriptions to buy online hydrocodone test urine buy fioricet i need to buy hydrocodone without prescription keyword buy hydrocodone keyword buy hydrocodone cod boards blogger help hydrocodone withdrawal buy hydrocodone tramadol buy tramadol now hydrocodone test urine buy phentermine hydrocodone test urine buy tramadol now hydrocodone tramadol buy now tramadol hydrocodone codeine buy online buy hydrocodone cod no membership buy domain hydrocodone b6 buy domain hydrocodone tripod buy domain hydrocodone tripod com buy drug hydrocodone online prescription buy delivery hydrocodone no online overnight presc buy consultation free hydrocodone online tramadol buy cod hydrocodone online buy cod hydrocodone overnight buy consult free hydrocodone buy drugs online heroin hydrocodone buy hydrocodone a buy hydrocodone cod online buy hydrocodone delaware buy hydrocodone direct doctors consultation on line buy hydrocodone don t have medical records buy hydrocodone cheap 24hr delivery buy hydrocodone cash delivery buy hydrocodone ao buy hydrocodone apap same day fedex shipping buy hydrocodone buy now tramadol buy cod hydrocodone no payment prescription shipping ups buy cod hydrocodone cheap tramadol b buy b hydrocodone b online b bu hydrocodone online buy now tramadol bu hydrocodone online buy tramadol now buy buy hydrocodone apap hydrocodone w buy now tramadol abuse hydrocodone buy viagra 5 500 apap hydrocodone buy tramadol now 853 hydrocodone watson buy now tramadol abuse hydrocodone buy soma buy buy hydrocodone online buy by cod hydrocodone online buy cheapest hydrocodone buy cod hydrocodone buy now tramadol buy cod hydrocodone buy tramadol now buy cheap hydrocodone online buy hydrocodone line buy cheap hydrocodone free consultation buy card credit hydrocodone online without buy card discover hydrocodone no overnight prescription buy cheap hydrocodone cod buy hydrocodone free consult buy hydrocodone in kentucky buy hydrocodone pill buy hydrocodone prescription records buy hydrocodone prescription tramadol on line buy hydrocodone related buy hydrocodone overseas buy hydrocodone overnight buy hydrocodone online with free consult buy hydrocodone online without a prescription cod buy hydrocodone onlineaaaa buy hydrocodone shark scare couple relive ordeal buy hydrocodone to shipped in arkansas buy hydrocodone with out perscription buy hydrocodone with perscription buy hydrocodone without a perscription buy hydrocodone without a prescrption buy hydrocodone with check buy hydrocodone with a perscription buy hydrocodone vicodin online cheap cod buy hydrocodone vicodin purchase pain buy hydrocodone vicoprofen buy hydrocodone online sent cod buy hydrocodone online rx without buy hydrocodone no consult buy hydrocodone no consultation fee buy hydrocodone no credit cards needed buy hydrocodone no medical record buy hydrocodone necessary no online prescription buy hydrocodone lortab medicine online buy hydrocodone khodorkovsky appeal to be heard buy hydrocodone lorcet lortab buy hydrocodone lortab codiene buy hydrocodone no medical records no prescription buy hydrocodone no online prescription buy hydrocodone online legally buy hydrocodone online no rx buy hydrocodone online overseas online consultations buy hydrocodone online deliver missouri buy hydrocodone online ao buy hydrocodone no presciption buy hydrocodone no presription buy hydrocodone online and pharmacy buy hydrocodone without contacting your doctor hydrocodone side effects side effects of hydrocodone hydrocodone effects side effects to hydrocodone side effects of hydrocodone cough medicine hydrocodone side effects of withdrawal does hydrocodone cause side effects hydrocodone compound syrup side effects side effects for hydrocodone hydrocodone long term side effects side effects hydrocodone effects hydrocodone side wapap depakote and hydrocodone what are the effects effects of expired hydrocodone hydrocodone acetaminophen side effects tablet vicodin hydrocodone com side effects of hydrocodine hydrocodone apap solution mck side effects hydrocodone here hydrocodine side effects snort hydrocodone side effects hydrocodone apap side effects side effects of hydrocodone used with fentanyl patch hydrocodone withdrawal effects addiction effects hydrocodone long term effects of longtime hydrocodone use effects hydrocodone side effects hydrocodone appetite effects of hydrocodone overdose effects of hydrocodone on the liver hydrocodone vicodin side effects ic hydrocodone side effects hydrocodone syrup side effects long term side effects of hydrocodone keyword hydrocodone side effects qoclick hydrocodone side effects mood changes withdrawals hydrocodone effects over time hydrocodone effects on fetus hydrocodone effects ten hours after m361 hydrocodone effects hydrocodone sie effects hydrocodone sise effects side effects of hydrocodone cough syrup addiction the effects of taking out of date hydrocodone what are the side effects of hydrocodone withdrawal effects of hydrocodone hydrocodone effects of long term abuse the effects of snorting hydrocodone the effects of hydrocodone with oxycodone side effects of hydrocodone 3f side effects of hydrocodone with long term use sign effects of hydrocodone snorting hydrocodone side effects side effects for hydrocodone compound syrup how to increase hydrocodone effects effects of hydrocodone guaifen effects of hydrocodone on fetus effects of hydrocodone pills hydrocodone side effects effects of hydrocodone and how long does it stay in effects of hydrocodone addiction hydrocodone 10650 side effects hydrocodone com hydrocodine side effects hydrocodone compound syrup mor side effects hydrocodone dosage side effects hydrocodone effects and side effects hydrocodone com effects of hydrocodone apap 5 500cap mck ingredients and effects hydrocodone abuse effects hydrocodone addiction respiratory effects hydrocodone apap 10mg 500mg side effects hydrocodone effects if snorted hydrocodone online hydrocodone apap no prescription hydrocodone hydrocodone no prescription hydrocodone withdrawal hydrocodone bitartrate hydrocodone without prescription hydrocodone online prescription how long does hydrocodone stay in your system norco hydrocodone what is hydrocodone hydrocodone without a prescription hydrocodone cod hydrocodone drugs hydrocodone dosage online prescription for hydrocodone hydrocodone prescription for pain pain drug hydrocodone hydrocodone overdose hydrocodone and pregnancy online pharmacy hydrocodone hydrocodone pictures order hydrocodone hydrocodone m358 hydrocodone overnight hydrocodone order online hydrocodone extraction pictures of hydrocodone online hydrocodone hydrocodone drug test hydrocodone detox ordering hydrocodone without a prescription hydrocodone withdrawal symptoms snorting hydrocodone foreign pharmacy hydrocodone no prescription hydrocodone no rx purchase hydrocodone cod withdrawal hydrocodone symptoms of hydrocodone addiction oxycodone vs hydrocodone hydrocodone online consultation hydrocodone apap 5 hydrocodone detection limits watson hydrocodone tablets hydrocodone oxycodone somas and hydrocodone prescriptions online pain meds hydrocodone hydrocodone for sale cold water extraction hydrocodone hydrocodone and no prescrition needed hydrocodone strength hydrocodone prescriptions hydrocodone withdrawl hydrocodone no acetaminophen order hydrocodone online nrop hydrocodone hydrocodone on line hydrocodone apap fact sheet hydrocodone m367 hydrocodone codeine gallbladder hydrocodone hydrocodone forums hydrocodone cold water extraction hydrocodone stays in system hydrocodone online no prescription hydrocodone vs oxycodone what is the drug hydrocodone used for hydrocodone no prescription call consult no prescription hydrocodone online hydrocodone mexico free online prescriptions for hydrocodone cheap hydrocodone hydrocodone compound syrup 20 mg hydrocodone capsule hydrocodone lorcet hydrocodone during pregnancy hydrocodone no prescription cod option hydrocodone for sale from foreign pharmacies hydrocodone prescription hydrocodone ingredient mexico hydrocodone hydrocodone detoxification vicodin hydrocodone bitartrate m357 hydrocodone watson hydrocodone doctor consultations hydrocodone cheep paris hydrocodone hydrocodone cough syrup no prescription hydrocodone 10mg how to order hydrocodone from cananda hydrocodone cough syrup hydrocodone cod no perscription hydrocodone abuse hydrocodone w apap hydrocodone tapering hydrocodone m357 hydrocodone picture what is stronger methadone or hydrocodone hydrocodone ingredients what does hydrocodone look like time between hydrocodone vicodin refills no perscription hydrocodone hydrocodone overseas cheep paris france hydrocodone generic hydrocodone cheapest online hydrocodone hydrocodone half life common dosage of hydrocodone pills hydrocodone cod free consutation hydrocodone 5 hydrocodone blood sugar hydrocodone drug hydrocodone acetaminophen hydrocodone vicodin hydrocodone withdrawal treatment hydrocodone v1096 hydrocodone prescription doctor liquid hydrocodone hydrocodone tolerance what is the appropriate dose of hydrocodone purchase hydrocodone what is hydrocodone prescribe for oxycontin hydrocodone oxycodone hydrocodone online prescriptions for somas and hydrocodone meperidine vs hydrocodone hydrocodone no medical records can hydrocodone and cocaine be confused on a urine test forigen hydrocodone no prescription blue hydrocodone 650 pictures how long does hydrocodone stay in your system for hydrocodone online cod hydrocodone medication hydrocodone high hydrocodone 5mg hydrocodone 10325 hydrocodone no prior prescription required hydrocodone no prescription cod hydrocodone uses hydrocodone money orders hydrocodone syrup hydrocodone sale hydrocodone lortab hydrocodone kentucky hydrocodone with apap hydrocodone liquid hydrocodone m360 hydrocodone m363 hydrocodone milligram dosages online pharmacies selling hydrocodone pure hydrocodone online pharmacy sell hydrocodone sniffing hydrocodone taking hydrocodone while pregnant hydrocodone internet pharmacies m367 hydrocodone buy phentermine phentermine cheap where can i get the cheapest phentermine order phentermine online and cod shipping can i order phentermine online phentermine no consultation buy phentermine online buy phentermine cod buy cheap phentermine buy phentermine no prescription buy phentermine without prescription buy phentermine saturday delivery ohio buy phentermine without a prescription where can i buy phentermine buy phentermine in the uk phentermine buy buy phentermine online without prescription cheapest place to buy phentermine buy phentermine at amide pharmaceutical buy cheap phentermine online buy phentermine on line buy phentermine with mastercard phentermine to buy buy phentermine shipped usps buy phentermine fedex where to buy phentermine can i buy phentermine anywhere in uk buy phentermine online cash on delivery phentermine buy online buy phentermine in canada buy phentermine online with paypal where can i buy fda approved phentermine cod buy phentermine in kentucky buy phentermine online us pharmacy discover card ok buy phentermine online without a prescription buy phentermine online payment method cod accepted buy phentermine cheap ship to all 50 states buy phentermine mastercard buy phentermine with no prescription buy phentermine generic by amide buy phentermine with saturday delivery buy phentermine from foreign pharmacy buy phentermine fedex overnight buy eon labs phentermine buy phentermine collect on delivery buy phentermine online cheap buy phentermine online no prescription hydrocodone line buy phentermine phentermine buy cheapest buy phentermine prozac buy phentermine no processing fees recurring charges buy phentermine online with check buy cheap phentermine free fedex buy phentermine for uk buy cheap phentermine today best buy phentermine buy phentermine by cod buy cod pay phentermine buy phentermine overnight cod buy phentermine tablets buy phentermine without perscription buy phentermine online with mastercard buy phentermine yellow buy phentermine without doctor approval buy phentermine online cod buy phentermine on line without perceptions buy phentermine on line without persciptions buy viagra phentermine online pharmacy buy phentermine online no hassle buy phentermine online same day delivery is it illegal to buy phentermine over the internet phentermine buy with american express phentermine buy cheapest online where can i buy phentermine online where to buy online phentermine you can still buy phentermine phentermine buy cheap phentermine buy hydrocodone liquid buy phentermine hydrocodone test urine buy phentermine buy phentermine now online no prescription pharmacy buy phentermine cod extraction hydrocodone buy phentermine buy phentermine in buy cheap phentermine yellow buy cheap pharmacy phentermine usa buy cod phentermine buy day phentermine same shipping buy domain onlinebigsitecitycom phentermine buy cheap no phentermine prescription best site to buy phentermine buy cheap diet phentermine pill buy cheap domain onlinemiheyorg phentermine phentermine buy cheap link myblogde phentermine buy eon phentermine buy florida in phentermine buy phentermine diet pill buy phentermine free consultation buy phentermine low cost buy phentermine no fees buy phentermine cheap overseas buy phentermine cheap buy generic phentermine buy pal pay phentermine using buy phentermine 30mg buy phentermine and hydrocodone com buy phentermine no rx cheap phentermine free shipping extra cheap phentermine cheap 37 5 phentermine cheap phentermine online cheap phentermine no rx cheap prices phentermine cheap phentermine canada phentermine cheap free shipping cheap phentermine free consult cheap phentermine without prescription free shipping cheap phentermine cheap phentermine no prescription phentermine cheap no prescription cheap phentermine overnight delivery cheap phentermine overnight cheap phentermine online 37 5 cheap phentermine without a prescription cheap phentermine paypal cheap phentermine pills cheap phentermine with online consultation cheap discount phentermine phentermine very cheap cheap phentermine online withour prescription cheap phentermine online no prescription cheap phentermine no perscription cheap online phentermine cheap perscription phentermine phentermine less cheap phentermine online cheap phentermine 15 mg cheap no prescription saturday delivery cheap phentermine cheap phentermine cod cheap phentermine without rx cheap overnight phentermine but phentermine cheap overseas cheap phentermine online without a prescription cheap phentermine using mastercard cheap phentermine shipped to kentucky cheap phentermine perscription phentermine cheap on line free shipping phentermine for cheap cheap phentermine without prior prescription 180 count cheap phentermine next day cheap phentermine pharmacies overseas xanax phentermine cheap phentermine 180 cheap no prescription extra cheap phentermine saturday delivery cheap yellow phentermine cheap phentermine with cod cheap phentermine withour prescription phentermine 30mg cheap no prescription ky cheap prescription phentermine extra cheap phentermine pay by check online phentermine cheap canada phentermine cheap online phentermine diet pills cheap phentermine low price cheap pills cheap phentermine phentermine cheap no perscription phentermine cheap free consultation cheap phentermine shipped to louisiana phentermine cheap cod phentermine cheap cod fedex phentermine 30mg cheap no prescription ky yellow cheap phentermine order online cheap adipex p or phentermine cheap domain ebuy online com phentermine cheap phentermine no rx in stock cheap phentermine or adipex online withour prescription cheap phentermine perscriptions 180 count cheap phentermine saturday delivery cheap phentermine for less cheap phentermine florida no prescription cheap internet phentermine suppliers cheap link onlinesplindercom phentermine cheap overnite phentermine cheap perscription diet pills phentermine cheap phentermine saturday delivery ups cheapest phentermine online cheapest phentermine pills phentermine cheapest price saturday delivery cheapest phentermine cheapest phentermine 90 day orders phentermine cheapest cheapest price phentermine online pharmacy cheapest free shipping phentermine online phentermine pharmacy best cheapest cheapest phentermine online no prescription cheapest phentermine online free shipping cheapest phentermine overnight cheapest phentermine prices ups saturday cheapest phentermine cheapest phentermine by check cheapest mail order phentermine phentermine cheapest online cheapest phentermine diet pills cheapest phentermine on line no prescription cheapest phentermine sent to arkansas phentermine 90 day cheapest fedx overnight cheapest phentermine phentermine cheapest cod the cheapest phentermine on line phentermine cheapest reputable cheapest phentermine recurring charges cheapest phentermine fast cheapest phentermine diet drugs cheapest phentermine next day cheapest phentermine no membership fee cheapest phentermine on line cheapest price for phentermine phentermine pill online discount discount online phentermine discount phentermine to florida phentermine discount discount phentermine no doctors prescription phentermine pill discount phentermine at discount prices discount phentermine no prescription no prescription discount phentermine phentermine no prescription discount phentermine sale discount discount phentermine without doctor contact discount phentermine prescription discount phentermine discount phentermine free shipping discount phentermine online order phentermine online order phentermine no prescription order phentermine online uk phentermine order phentermine 6 pm order phentermine order easy phentermine money order order phentermine by for saturday delivery order phentermine cod order phentermine online without perscription order phentermine by cod phentermine pay by money order no hassle order 100 phentermine leo phentermine order online online phentermine order order phentermine online cod order phentermine on line phentermine pod money order phentermine check money order order phentermine online without prescription order and phentermine a website to order phentermine that will send to ky cod order phentermine phentermine pay by florida money order phentermine online order 3pm phentermine secure online order where can i order phentermine online why phentermine pay by money order order with money orders phentermine where can i order phentermine phone order only phentermine order phentermine overnight check online order personal phentermine 00 6 by order p m phentermine money order phentermine order phentermine c o d order phentermine pay cod order phentermine diet pill order phentermine uk phentermine 37 5mg phentermine diet pills phentermine online online phentermine phentermine no prescription phentermine side effects phentermine without prescription phentermine on line phentermine free shipping phentermine without perscription no prescription phentermine phentermine cod phentermine hcl phentermine for sale how does phentermine work phentermine overnight herbal phentermine phentermine 30 mg phentermine diet pill phentermine on line without prescription phentermine mastercard phentermine without a prescription no prescripton phentermine phentermine shortage diet pill phentermine phentermine saturday delivery phentermine no perscription phentermine tablets phentermine yellow phentermine alternatives phentermine 30mg alternatives to phentermine what is phentermine alternative to phentermine online phentermine prescription phentermine no rx phentermine hydrochloride phentermine pills lowest price for phentermine phentermine forum phentermine accepts cod phentermine on line wo prescription adipex meridia phentermine xenical side effects of phentermine generic phentermine phentermine weight loss pills does phentermine work phentermine overnight delivery phentermine pill phentermine information phentermine sales phentermine next day phentermine info phentermine blue phentermine hci phentermine ingredients adipex phentermine low cost phentermine phentermine cash on delivery phentermine no prescription free shipping phentermine prices phentermine drug test phentermine diet pills diet pills phentermine blue diet pills dangers of phentermine ordering phentermine phentermine free consultation information on phentermine uk online pharmacy phentermine free shipping on phentermine diet pills cod phentermine fda us approved phentermine phentermine pharmacy phentermine with free shipping phentermine successful stories phentermine online without prescription phentermine no prescription required compare phentermine prices phentermine no credit card required phentermine results phentermine ship to ky phentermine prescription prescriptions on line phentermine phentermine with cod payments free shipping phentermine phentermine canada negative side effects of phentermine phentermine addiction phentermine prozac phentermine prescriptions phentermine abuse weight loss phentermine phentermine without rx phentermine vs phentrazine overnight phentermine online pharmacy and phentermine does phentermine speed up metabolism phentermine online consultation phentermine no prescription needed phentermine diet plan phentermine in stock compare ionamin phentermine phentermine gynecomastia phentermine cod delivery phentermine review purchase phentermine online without prescription phentermine phentermine side effects dangers lowest price phentermine with free shipping phentermine 15 mg long term side effects of phentermine lowest cost phentermine phentermine online pharmacy phentermine np phentermine pharmacies online phentermine reviews side effects phentermine gynecomastia phentermine price phentermine no prescription required free shipping 100 phentermine pharmacy online phentermine fast phentermine does phentermine really work phentermine eprescriptions phentermine diet phentermine meridia xenical review phentermine dosage time released phentermine phentermine and fast shipping what does phentermine look like phentermine tolerance phentermine from mexico is there a phentermine shortage how much weight will you lose on phentermine phentermine a159 lowest phentermine prices drug interaction sibutramine and phentermine phentermine 37_5 overnight phentermine shipping phentermine success stories phentermine no perscription required phentermine tricks purchase phentermine phentermine mexico research phentermine tolerance phentermine us pharmacy online consultation phentermine on sale cause rebound weight gain phentermine best phentermine pharmacies compare links chep phentermine nonprescription phentermine phentermine 89 phentermine overnight saturday delivery phentermine chemical enhancement phentermine and diet pill phentermine priority mail phentermine and atkins diet shipping phentermine to florida what does phentermine do to your heart phentermine without prescription and energy pill phentermine withdrawal phentermine with no prescription phentermine airborne express phentermine rankings how fast will phentermine work phentermine adipex i need to lose 15 lbs phentermine paxil with phentermine interactions online consultations and prescriptions phentermine online pharmacy phentermine phentermine florida phentermine no credit card cod phentermine no fees phentermine ionamin phentermine online no prescription phentermine at cost with no prescription phentermine by cod phentermine dangers phentermine drops phentermine weight loss phentermine with master card real phentermine weight loss oral hcg and phentermine what is phentermine civ phentermine us phentermine purchase phentermine risks phentermine pill town phentermine feedback diet pills phentermine adipex phentermine weight loss best online deal for phentermine black market phentermine different types of phentermine phentermine 37 5mg and mastercard does phentermine help weight loss paris cheep phentermine phentermine 15 mgs overnight phentermine no prescription fda approved phentermine next day phentermine i need to lose 10 lbs phentermine buy phentermine phentermine cheap where can i get the cheapest phentermine order phentermine online and cod shipping can i order phentermine online phentermine no consultation buy phentermine online buy phentermine cod buy cheap phentermine buy phentermine no prescription buy phentermine without prescription buy phentermine saturday delivery ohio buy phentermine without a prescription where can i buy phentermine buy phentermine in the uk phentermine buy buy phentermine online without prescription cheapest place to buy phentermine buy phentermine at amide pharmaceutical buy cheap phentermine online buy phentermine on line buy phentermine with mastercard phentermine to buy buy phentermine shipped usps buy phentermine fedex where to buy phentermine can i buy phentermine anywhere in uk buy phentermine online cash on delivery phentermine buy online buy phentermine in canada buy phentermine online with paypal where can i buy fda approved phentermine cod buy phentermine in kentucky buy phentermine online us pharmacy discover card ok buy phentermine online without a prescription buy phentermine online payment method cod accepted buy phentermine cheap ship to all 50 states buy phentermine mastercard buy phentermine with no prescription buy phentermine generic by amide buy phentermine with saturday delivery buy phentermine from foreign pharmacy buy phentermine fedex overnight buy eon labs phentermine buy phentermine collect on delivery buy phentermine online cheap buy phentermine online no prescription hydrocodone line buy phentermine phentermine buy cheapest buy phentermine prozac buy phentermine no processing fees recurring charges buy phentermine online with check buy cheap phentermine free fedex buy phentermine for uk buy cheap phentermine today best buy phentermine buy phentermine by cod buy cod pay phentermine buy phentermine overnight cod buy phentermine tablets buy phentermine without perscription buy phentermine online with mastercard buy phentermine yellow buy phentermine without doctor approval buy phentermine online cod buy phentermine on line without perceptions buy phentermine on line without persciptions buy viagra phentermine online pharmacy buy phentermine online no hassle buy phentermine online same day delivery is it illegal to buy phentermine over the internet phentermine buy with american express phentermine buy cheapest online where can i buy phentermine online where to buy online phentermine you can still buy phentermine phentermine buy cheap phentermine buy hydrocodone liquid buy phentermine hydrocodone test urine buy phentermine buy phentermine now online no prescription pharmacy buy phentermine cod extraction hydrocodone buy phentermine buy phentermine in buy cheap phentermine yellow buy cheap pharmacy phentermine usa buy cod phentermine buy day phentermine same shipping buy domain onlinebigsitecitycom phentermine buy cheap no phentermine prescription best site to buy phentermine buy cheap diet phentermine pill buy cheap domain onlinemiheyorg phentermine phentermine buy cheap link myblogde phentermine buy eon phentermine buy florida in phentermine buy phentermine diet pill buy phentermine free consultation buy phentermine low cost buy phentermine no fees buy phentermine cheap overseas buy phentermine cheap buy generic phentermine buy pal pay phentermine using buy phentermine 30mg buy phentermine and hydrocodone com buy phentermine no rx cheap phentermine free shipping extra cheap phentermine cheap 37 5 phentermine cheap phentermine online cheap phentermine no rx cheap prices phentermine cheap phentermine canada phentermine cheap free shipping cheap phentermine free consult cheap phentermine without prescription free shipping cheap phentermine cheap phentermine no prescription phentermine cheap no prescription cheap phentermine overnight delivery cheap phentermine overnight cheap phentermine online 37 5 cheap phentermine without a prescription cheap phentermine paypal cheap phentermine pills cheap phentermine with online consultation cheap discount phentermine phentermine very cheap cheap phentermine online withour prescription cheap phentermine online no prescription cheap phentermine no perscription cheap online phentermine cheap perscription phentermine phentermine less cheap phentermine online cheap phentermine 15 mg cheap no prescription saturday delivery cheap phentermine cheap phentermine cod cheap phentermine without rx cheap overnight phentermine but phentermine cheap overseas cheap phentermine online without a prescription cheap phentermine using mastercard cheap phentermine shipped to kentucky cheap phentermine perscription phentermine cheap on line free shipping phentermine for cheap cheap phentermine without prior prescription 180 count cheap phentermine next day cheap phentermine pharmacies overseas xanax phentermine cheap phentermine 180 cheap no prescription extra cheap phentermine saturday delivery cheap yellow phentermine cheap phentermine with cod cheap phentermine withour prescription phentermine 30mg cheap no prescription ky cheap prescription phentermine extra cheap phentermine pay by check online phentermine cheap canada phentermine cheap online phentermine diet pills cheap phentermine low price cheap pills cheap phentermine phentermine cheap no perscription phentermine cheap free consultation cheap phentermine shipped to louisiana phentermine cheap cod phentermine cheap cod fedex phentermine 30mg cheap no prescription ky yellow cheap phentermine order online cheap adipex p or phentermine cheap domain ebuy online com phentermine cheap phentermine no rx in stock cheap phentermine or adipex online withour prescription cheap phentermine perscriptions 180 count cheap phentermine saturday delivery cheap phentermine for less cheap phentermine florida no prescription cheap internet phentermine suppliers cheap link onlinesplindercom phentermine cheap overnite phentermine cheap perscription diet pills phentermine cheap phentermine saturday delivery ups cheapest phentermine online cheapest phentermine pills phentermine cheapest price saturday delivery cheapest phentermine cheapest phentermine 90 day orders phentermine cheapest cheapest price phentermine online pharmacy cheapest free shipping phentermine online phentermine pharmacy best cheapest cheapest phentermine online no prescription cheapest phentermine online free shipping cheapest phentermine overnight cheapest phentermine prices ups saturday cheapest phentermine cheapest phentermine by check cheapest mail order phentermine phentermine cheapest online cheapest phentermine diet pills cheapest phentermine on line no prescription cheapest phentermine sent to arkansas phentermine 90 day cheapest fedx overnight cheapest phentermine phentermine cheapest cod the cheapest phentermine on line phentermine cheapest reputable cheapest phentermine recurring charges cheapest phentermine fast cheapest phentermine diet drugs cheapest phentermine next day cheapest phentermine no membership fee cheapest phentermine on line cheapest price for phentermine phentermine pill online discount discount online phentermine discount phentermine to florida phentermine discount discount phentermine no doctors prescription phentermine pill discount phentermine at discount prices discount phentermine no prescription no prescription discount phentermine phentermine no prescription discount phentermine sale discount discount phentermine without doctor contact discount phentermine prescription discount phentermine discount phentermine free shipping discount phentermine online order phentermine online order phentermine no prescription order phentermine online uk phentermine order phentermine 6 pm order phentermine order easy phentermine money order order phentermine by for saturday delivery order phentermine cod order phentermine online without perscription order phentermine by cod phentermine pay by money order no hassle order 100 phentermine leo phentermine order online online phentermine order order phentermine online cod order phentermine on line phentermine pod money order phentermine check money order order phentermine online without prescription order and phentermine a website to order phentermine that will send to ky cod order phentermine phentermine pay by florida money order phentermine online order 3pm phentermine secure online order where can i order phentermine online why phentermine pay by money order order with money orders phentermine where can i order phentermine phone order only phentermine order phentermine overnight check online order personal phentermine 00 6 by order p m phentermine money order phentermine order phentermine c o d order phentermine pay cod order phentermine diet pill order phentermine uk phentermine 37 5mg phentermine diet pills phentermine online online phentermine phentermine no prescription phentermine side effects phentermine without prescription phentermine on line phentermine free shipping phentermine without perscription no prescription phentermine phentermine cod phentermine hcl phentermine for sale how does phentermine work phentermine overnight herbal phentermine phentermine 30 mg phentermine diet pill phentermine on line without prescription phentermine mastercard phentermine without a prescription no prescripton phentermine phentermine shortage diet pill phentermine phentermine saturday delivery phentermine no perscription phentermine tablets phentermine yellow phentermine alternatives phentermine 30mg alternatives to phentermine what is phentermine alternative to phentermine online phentermine prescription phentermine no rx phentermine hydrochloride phentermine pills lowest price for phentermine phentermine forum phentermine accepts cod phentermine on line wo prescription adipex meridia phentermine xenical side effects of phentermine generic phentermine phentermine weight loss pills does phentermine work phentermine overnight delivery phentermine pill phentermine information phentermine sales phentermine next day phentermine info phentermine blue phentermine hci phentermine ingredients adipex phentermine low cost phentermine phentermine cash on delivery phentermine no prescription free shipping phentermine prices phentermine drug test phentermine diet pills diet pills phentermine blue diet pills dangers of phentermine ordering phentermine phentermine free consultation information on phentermine uk online pharmacy phentermine free shipping on phentermine diet pills cod phentermine fda us approved phentermine phentermine pharmacy phentermine with free shipping phentermine successful stories phentermine online without prescription phentermine no prescription required compare phentermine prices phentermine no credit card required phentermine results phentermine ship to ky phentermine prescription prescriptions on line phentermine phentermine with cod payments free shipping phentermine phentermine canada negative side effects of phentermine phentermine addiction phentermine prozac phentermine prescriptions phentermine abuse weight loss phentermine phentermine without rx phentermine vs phentrazine overnight phentermine online pharmacy and phentermine does phentermine speed up metabolism phentermine online consultation phentermine no prescription needed phentermine diet plan phentermine in stock compare ionamin phentermine phentermine gynecomastia phentermine cod delivery phentermine review purchase phentermine online without prescription phentermine phentermine side effects dangers lowest price phentermine with free shipping phentermine 15 mg long term side effects of phentermine lowest cost phentermine phentermine online pharmacy phentermine np phentermine pharmacies online phentermine reviews side effects phentermine gynecomastia phentermine price phentermine no prescription required free shipping 100 phentermine pharmacy online phentermine fast phentermine does phentermine really work phentermine eprescriptions phentermine diet phentermine meridia xenical review phentermine dosage time released phentermine phentermine and fast shipping what does phentermine look like phentermine tolerance phentermine from mexico is there a phentermine shortage how much weight will you lose on phentermine phentermine a159 lowest phentermine prices drug interaction sibutramine and phentermine phentermine 37_5 overnight phentermine shipping phentermine success stories phentermine no perscription required phentermine tricks purchase phentermine phentermine mexico research phentermine tolerance phentermine us pharmacy online consultation phentermine on sale cause rebound weight gain phentermine best phentermine pharmacies compare links chep phentermine nonprescription phentermine phentermine 89 phentermine overnight saturday delivery phentermine chemical enhancement phentermine and diet pill phentermine priority mail phentermine and atkins diet shipping phentermine to florida what does phentermine do to your heart phentermine without prescription and energy pill phentermine withdrawal phentermine with no prescription phentermine airborne express phentermine rankings how fast will phentermine work phentermine adipex i need to lose 15 lbs phentermine paxil with phentermine interactions online consultations and prescriptions phentermine online pharmacy phentermine phentermine florida phentermine no credit card cod phentermine no fees phentermine ionamin phentermine online no prescription phentermine at cost with no prescription phentermine by cod phentermine dangers phentermine drops phentermine weight loss phentermine with master card real phentermine weight loss oral hcg and phentermine what is phentermine civ phentermine us phentermine purchase phentermine risks phentermine pill town phentermine feedback diet pills phentermine adipex phentermine weight loss best online deal for phentermine black market phentermine different types of phentermine phentermine 37 5mg and mastercard does phentermine help weight loss paris cheep phentermine phentermine 15 mgs overnight phentermine no prescription fda approved phentermine next day phentermine i need to lose 10 lbs phentermine