Direct Digital Synthesis

The New Way To Generate Sine Waves

In the good old days before everything started going digital, sine wave generators were in the domain of analog circuitry. One built them using Hartley, Colpitts, Pierce, and other such oscillator circuit designs. They depended on variable capacitors or varactor diodes for tuning, and required good construction and shielding practices to achieve acceptable frequency stability. But all good things come to an end. The era of digital signal processing has arrived, and along with it all sorts of new digital and mixed signal devices.

The Direct Digital Synthesizer (DDS) is simple in concept: one stores a table of sine curve values (one cycles worth of values) which are consecutively read from the table and converted to a corresponding voltage. When the end of the table is reached, the process is repeated. Over time, the voltage output generates a continuous sine wave. Changing the rate at which the samples are read and converted changes the frequency of the sine wave. What could be easier?

Unfortunately, a variety of demons await the unsuspecting designer. To see why, we must define the process a little more carefully. Each voltage output is called a ’sample’. The number of samples per second is known as the sampling rate. Suppose there are N entries in the table for one cycle of the sine wave. The output frequency of the sine wave, f, will then be the sampling rate, Fs, divided by N.

                                                                 f = Fs/N

The voltage output of the DDS is generated by a digital to analog converter. After a sample is output, the voltage remains constant until the next sample is output. The resulting sine wave has a stair-step quality to its shape. The smoothness of the sine wave depends upon the number of entries in the table, and the number of bits of precision of each entry (or the number of bits the digital to analog converter can handle).

The stair-step nature of the sine wave means it is not a pure, single, frequency sine wave. We might guess that the non-sinusoidal nature indicates there will be harmonics present (integer multiple frequencies of the desired frequency), however, this turns out not to be the case. The frequency spectrum is a bit more complicated. Suppose the sampling rate is 10 MHz, and our sine wave is 1 MHz. This means our table has 10 samples for each cycle. Analysis of the frequency content of our stair-step sine wave shows our desired 1 MHz signal, but it also includes a number of other frequencies almost as prominent as the one we want, namely: 9 MHz, 11 MHz, 19 MHz, 21 MHz, 29 MHz, etc. The pattern here is pretty obvious: every 10 MHz frequency segment includes 2 output frequencies, one that is 1 MHz up from the bottom end, and one that is one MHz down from the top end. To produce a “pure” sine wave, a good quality filter must be used to eliminate all but the desired frequency. Typically, this is a low pass filter, often containing 4 poles or more. A clever designer, however, can use a band pass filter tuned to any one of the the output frequencies. Note that this limits the useful output range of the DDS. For example, with a sample frequency again of 10 MHz, but a desired output of 4 MHz, the actual outputs will be at 4, 6, 14, 16, etc. To obtain a pure 4 MHz, the 6 MHz must be removed. This requires a very good filter.

Rather than increasing the sample rate to increase the output frequency, the DDS devices provide a means to select every other sample from the table (count by 2s), or every third sample, etc. This runs through the table faster converting fewer table values per cycle. If the table only includes 10 samples, this is not too useful, but if it holds 10,000,000 samples for a cycle, this is very useful. At each sample time, the user specified increment, n, is added to an internal ‘phase counter’ to keep track of where to look in the sample table. Modern devices use compressed tables along with clever mathematical techniques that produce effective table sizes of billions of samples for one cycle of a sine wave. The output frequency is now given by:

                                                                 f = n x Fs / N

Other sources of difficulty lie in wait for those seeking ultra pure sine waves. Inaccuracies in the analog to digital converter produce unwanted frequencies called ’spurs’, as does the conversion process when the phase counter is truncated to the number of bits used by the analog to digital converter. These spurs, usually of pretty low level, produce ‘phase jitter’ in the resulting sine wave, which may cause difficulties in some applications. Techniques exist to mitigate some of these effects, but designers of high performance systems must be aware of them and the limitations they may impose.

Modern DDS devices use high frequency clock rates. For example, the Analog Devices AD9851 operates with a sample rate of 180 MHz, and can generate sine waves up to 90 MHz, but in practice 60 MHz is a useful upper limit (the first extra output is then at 120 MHz and can be removed by a low pass filter). Other devices are available with much lower, or much higher sample rates.

These devices are rapidly moving from the sphere of the esoteric to the common place of the electronics hobbyist. I recently obtained a DDS kit based on the AD9851 as pictured below. As can be seen, the parts are mostly surface mount and require appropriate tools and a steady hand to solder them to the board. The IC on the left hand side of the left hand picture is the AD9851. This board will function as a general purpose r.f. signal generator, a frequency source for a direct conversion ham receiver, and a vfo for a ham transmitter. Since its accuracy and stability are determined directly by the oscillator module, it will operate with very little temperature drift and with high immunity to mechanical instability.


DDS devices are clever and ingenious devices. Because of their many superior capabilities, and smooth integration into digital systems, their popularity is sure to increase. For more information, I recommend the following sources:

http://www.analog.com/en/rfif-components/direct-digital-synthesis-dds/ad9851/products/product.html
http://www.amqrp.org/kits/dds60/index.html

Mixed Integer Programming (MIP)

Integer variables and Linear Programming equals Mixed Integer Programming

Linear Programming has long been an important optimization tool, especially in the business arena. Decisions regarding optimum inventory quantities, shipping and transportation choices, manufacturing scheduling, and numerous other processes can often benefit from the application of Linear Programming (LP).

In its simplest form, linear programming is a technique for solving a set of mathematical inequalities subject to a particular optimization goal. Consider the following optimization problem: a company sells two products, A and B, and wants to make as much profit each week as possible. The company is just getting started and has space constraints for inventory, and budget constraints for both inventory purchase and labor.

If a and b represent the quantity of products A and B they can produce each week, the following constraints must be met:

inventory space:       2a + b <= 2000         since units of A take twice as much room as those of B,
inventory cost:            a + b <= 1400         since the unit materials cost the same amount,
labor cost:                        b <= 1000         since units of A require no labor,

and the profit is given by P = 3a + 2b, that is, units of A are more profitable.

This problem is directly solvable by standard ’simplex’ LP solvers. The solution to this problem is a = 600, b = 800, P = 3400, and the labor is idle 20% of the time, so a 4 day work week might be appropriate. Although a and b worked out to be integers in this problem, that is not always the case. But the answers, if decimal numbers, would still be close enough to be useful.

But now suppose that the inventory of A can only be purchased in lots of 250. What is the best solution?

The new constraint is a = 250c, where c must be an integer (the number of lots).

The LP solver does not allow integer constraints. Running it with this constraint would produce the same solution, with c = 2.4 (so that a = 250c = 600 as before.

This problem can still be solved, but it now requires use of a Mixed Integer Programming (MIP) solver. A MIP solver allows us to constrain the variable c to be an integer.

The new solution will be a = 500, b = 900, c = 2, P = 3300, with the labor idle only 10% of the time. These differences in the solution could be very important to the business.

In practice, these types of problems typically have many more variables and many more constraints, perhaps numbering in the hundreds or thousands.

As a final note, MIP solvers often take much longer to solve a problem than an LP solver, since the MIP solver runs iteratively, and each iteration calculates an LP solution. The art of MIP usage is knowing how to reduce the number of required iterations.

A good advanced text on MIP solver theory is “Production Planning by Mixed Integer Programming” by Pochet and Wolsey, published by Springer. Many high school algebra texts cover introductory graphical methods for the solution of simple LP problems.

Poker and Other Bluffing Games

In July of 2007, a computer program called Polaris played against two of the best human poker players, Phil Laak Four Acesand Ali Aslami, in a four day tournament of limit Texas Hold ‘em. Oddly for poker, the match was carefully designed so that the result could not be attributable to good or bad cards.

To do this, two copies of the program played on separate computers in separate rooms. One computer played against Phil Laak in one room and the other computer played against Ali Aslami in the other room. The decks were arranged so that the cards dealt in one room were identical to the cards dealt in the other room. However, the cards given to Polaris in one room were given to the human in the other, and vice versa. This ensured that if the cards favored Polaris in one room, then they favored the human in the other room by exactly the same amount (since they were the same cards). As a result, whether Polaris won or not, it could not be because it received good or bad cards.

The first day of the match was a draw. What this actually means is that Polaris won, but only by $70, and a win by less than $250 was considered statistically insignificant. The second day Polaris won, and the third and fourth days the humans won. Although Polaris lost the tournament, it put on an unquestionably good showing.

What is particularly interesting about this is that in order to win at poker, one must be able to bluff well, and it seems to run rather counterintuitive to most people’s opinions of machines that they should actually be able to bluff. This was highlighted in the Star Trek Next Generation episode “The Measure of a Man” when Riker bluffed Data1:

DATA: I will fold.
Riker rakes in the chips, then turns over his cards revealing a busted hand. Data turns his cards face up. He held a winning hand.
DATA: You had nothing.
GEORDI: He bluffed you, Data.
DATA: It makes very little sense to bet when you cannot win.
RIKER: But I did win. I was gambling that you wouldn’t call.
DATA: But how can you tell?
O’BRIEN: Instinct, Data, instinct.

I like Star Trek, but they got it wrong here. In fact, there is a mathematically perfect way to bluff, and no instinct is required at all. Unfortunately, perfect bluffing in a complex game like Texas Hold ‘Em requires more computational power than will ever be available. Fortunately, near perfect bluffing can be done on a desktop. It appears that the best humans bluff in a fashion that is still nearer perfect than a computer can yet achieve, but this is almost certain to reverse in the next few years (at least for two player limit Texas Hold ‘Em).

Unlike humans, who often bluff on instinct, a computer bluffs based on random chance. This ensures that its opponent cannot guess when it is bluffing. Of course, the probability of the computer bluffing is dependent on its cards, the opponent’s play, and so forth, but there are optimal values for these probabilities, and the “art” of bluffing really becomes a science.

1. Excerpted from Star Trek Minutiae

Scene Completion

Scene Completion Teaser

Scene completion is the task of filling in holes in a picture, and algorithms for doing it automatically are starting to become impressive. In the series of pictures above, the original picture (left) has an obnoxious roof taking up a lot of the picture. In the next picture, the user erased the roof, leaving a gaping hole in the picture. A current algorithm for filling in holes uses other similar pictures to figure out what should go in the hole, so the next picture shows similar pictures from a picture database. The final picture has the hole filled in with a reasonable guess.

The algorithm that generated the above sequence of pictures is due to James Hays and Alexei Efros and is explained in this paper. It is driven by a database of 2.3 million images of landscape, travel, and city photography. The first stage of the algorithm is to get the “gist” of the scene, which is a descriptor that can distinguish between cities, hillsides, beaches, and so forth. The algorithm uses this to find a set of images from its database that are of the same general type of thing. This prevents the algorithm from filling in hillsides with car roofs.

Once it has a set of images of similar things, it looks at the region around the hole and tries to find similar regions in the list of pictures. Next it chooses a small set of these best fitting regions and blends each of them back into the original hole, making a set of composite pictures without holes. Finally, it presents each of these composite pictures to the user so they can choose which one they like the best.

The algorithm works very well most of the time. However, it still has a few drawbacks. First, it is currently only trained on city, landscape, and travel pictures. As the range of pictures grows, the problem will become more difficult. Second, the program currently runs on a cluster of 15 machines, instead of a single desktop PC. Third, it still makes serious mistakes occasionally (they show a failure case in the article where the bottom half of a lady was filled in with what looks like a trash can). These drawbacks are serious enough to show that scene completion is not yet ready for the mass market, but the overriding message I took from the article was that it will be ready quite a bit sooner than I would have guessed.