Bespoke Bytes

Biscuits

2023-08-10T22:45:00-07:00

Making Biscuits

Biscuits are one of my favorite things to make, satisfying the urge to work with my hands and providing a reward for the labor at the end.

Basic Recipe

2 C all-purpose flour
1 T baking powder
2 t brown sugar
1/2 t cream of tartar
1/4 t salt
1/2 C butter
2/3 C milk

preheat oven to 450˚F
mix dry ingredients
cut in butter to a coarse crumb texture
mix in milk
gently work to smooth dough
flatten to about 1/4 inch thick
fold and shape to rough rectangle 1/2 inch thick. you can repeat this, but avoid over working the dough
cut into squares, placing on a baking sheet
bake until golden brown, about 10-12 minutes

Variations

Spam & Cheese

1 can spam, diced

4 oz shredded cheese

toss in the spam and cheese before adding the milk

Sweet Rockets

1 T Butter
1/4 C brown sugar
1 t cinnamon

melt the butter and mix well with the cinnamon and sugar
before the last fold, spread mixture over half of dough
rather then cutting into squares, use a rocket shaped cutter

Holiday Cards 2022

2023-02-26T18:16:00-08:00

In 2022 I decided to send holiday cards to friends and family. The easy way to do this would have been to go to the store and buy a few boxes of season appropriate cards. This of course makes sense, is inexpensive, and super easy. Of course this didn't happen...

Setting the Scene

Professionally I spend a lot of time working on projects where there are few tangible outputs, and even fewer where there are visually interesting outputs that I can share with anyone. For these cards I decided that they should be computer generated, and should be visually interesting. Computer generated doesn't really narrow things down, so I decided to merge hobbies and have a robot draw the cards. In this case the robot in question was a CNC engraver that's been modified to serve as a janky pen plotter.

The spindle of the engraver was swapped out for a pen holder that was made from parts of the world's poorest quality bench grinder.

On the software front the CNC machine is running GRBL and is controlled via LaserWeb4 running on a Raspberry Pi.

Coding the Card

The card needed to capture the holiday spirit, and what better way to do that than have some pointy trees? The trees were generated by identifying points along the base of the tree and the trunk, then drawing a line from one side the base to the trunk and back to the other side of the trunk. Numpy for most of the calculations and Matplotlib was used to render the card during development.

The tree is fairly simple, generating a pleasing design:

def string_art(width: float = 1, height: float = 1,
               n_steps: float = 20,
               base_x: float = 0., base_y: float = 0.,
               rotation=0) -> tuple[list[list[float]], list[list[float]]]:
    """
    Construct a string art tree.

    The height of the tree is measured from the top of the trunk, with the
    trunk extending 0.1 * height below the tree.

    The output is a two-element tuple with the first element containing a
    list of lists of the x positions for the drawing segments. The second
    element is the positions.

    :param width: The width of the tree in millimeters
    :param height: The height of the tree in millimeters above the trunk.
    :param n_steps: The number of lines making up the tree.
    :param base_x: The x location of the tree's origin.
    :param base_y: The y location of the tree's origin.
    :param rotation: The rotation of the tree in radians.
    :return: The line segments representing the tree.

    """
    segments_x = []
    segments_y = []
    thetas = list(np.linspace(0, np.pi / 2, num=n_steps, endpoint=True))

    # Construct the base tree lines
    # The drawing alternates direction. This reduces motion in the final drawing.
    even = True
    for theta in thetas:
        if even:
            segments_x.append([np.cos(theta), 0, -np.cos(theta)])
        else:
            segments_x.append([-np.cos(theta), 0, np.cos(theta)])
        even = not even
        segments_y.append([0, np.sin(theta), 0])
    # The Trunk
    trunk_depth = -0.1
    trunk_width = 0.1
    segments_x.append([-trunk_width, -trunk_width, trunk_width])
    segments_y.append([0, trunk_depth, trunk_depth])
    segments_x.append([trunk_width, trunk_width, -trunk_width])
    segments_y.append([trunk_depth, 0, 0])

    return transform_strokes(segments_x, segments_y,
                             width=width / 2.0, height=height,
                             base_x=base_x, base_y=base_y,
                             rotation=rotation)

Feeding the tree generator a range of values for n_steps makes a family of trees.

The other big element required for a card is words. In this case we can take a look at the history of technology, and we'll come across Hershey Fonts described in DTIC AD0662398: CALLIGRAPHY FOR COMPUTERS by Dr. Allen Vincent Hershey. While the Hershey fonts were originally aimed at vector cathode ray tube displays, they're well-suited for other vector display technologies. Since the plotter draws straight line segments it's really just a vector display so we can use these fonts.

Putting the string art and fonts together we can start to compose a card. The planned card is 114 millimeters wide by 152 millimeters tall, since that fits nicely into a 4x6 inch envelope. The code that generates the outside of the card is:

def make_outside_of_card(script_font, block_font):
    # pylint: disable=too-many-locals
    """
    Generate the outside of the card.
    :param script_font: The font definition for the script font
    :param block_font: The font definition for the block font.
    :return: The segments that define the card
    """
    x_segments = []
    y_segments = []

    # Front
    trees = ((100, 90, 45, 171, 46),  # width, height,n_lines, base_x, base_y
             (20, 18, 9, 140, 114),
             (10, 12, 5, 210, 95),
             (10, 25, 13, 130, 58),
             )

    for tree in trees:
        tree_x, tree_y = string_art(*tree)
        x_segments.extend(tree_x)
        y_segments.extend(tree_y)

    x_message, y_message = construct_message("Happy Holidays", script_font, center=True)
    x_message, y_message = transform_message(x_message, y_message, height=15, base_x=171, base_y=30)
    x_segments.extend(x_message)
    y_segments.extend(y_message)

    x_message, y_message = construct_message("202012", script_font, center=True)
    x_bbox, y_bbox = construct_letter_bboxes("202012", script_font, center=True)
    for cross_out in (3, 4):
        x_squiggle, y_squiggle = cross_out_bbox(x_bbox[cross_out], y_bbox[cross_out], cycles=3)
        x_message.extend(x_squiggle)
        y_message.extend(y_squiggle)
    x_message, y_message = transform_message(x_message, y_message, height=15, base_x=171, base_y=15)
    x_segments.extend(x_message)
    y_segments.extend(y_message)

    # Back
    x_url, y_url = construct_message("github.com/palbee/holiday_card_2022", block_font, center=True)
    x_url, y_url = transform_message(x_url, y_url, height=6, base_x=105, base_y=76,
                                     rotation=np.deg2rad(90))
    x_segments.extend(x_url)
    y_segments.extend(y_url)

    return x_segments, y_segments

The actual code is fairly direct, with an attempt to make the code reasonably conversational. The card outside was rendered using MatPlotLib to make sure the design was within bounds. Note that the front and back of the card are plotted together.

The inside of the card was constructed in a similar manner, and is shown below.

After the card designs were finalized they needed to be plotted. This was done by converting the line segments to gcode and sending them off to the plotter. The gcode for a small portion of the outside of the card is:

G0 X221.0 Y46.0 Z-14.5
G0 Z-16.25
G1 X171.0 Y46.0 F1000
G1 X121.0 Y46.0 F1000
G0 Z-14.5
G0 X121.03185871715041 Y46.0 Z-14.5
G0 Z-16.25
G1 X171.0 Y49.21231004550824 F1000
G1 X220.9681412828496 Y46.0 F1000
G0 Z-14.5
G0 X220.87260573051267 Y46.0 Z-14.5
G0 Z-16.25
G1 X171.0 Y52.420526487930914 F1000
G1 X121.12739426948733 Y46.0 F1000
G0 Z-14.5
G0 X121.28648491140513 Y46.0 Z-14.5
G0 Z-16.25
G1 X171.0 Y55.62056094086015 F1000
G1 X220.71351508859487 Y46.0 F1000
G0 Z-14.5
G0 X220.49107209404664 Y46.0 Z-14.5
G0 Z-16.25
G1 X171.0 Y58.80833544459566 F1000
G1 X121.50892790595336 Y46.0 F1000
G0 Z-14.5

The code above draws a total of 14 line segments, and takes about 12 minutes to complete the outside of the card. The inside of the card takes an additional 14 minutes, in part because the row of trees is fairly detailed. The text also adds significant complexity and time to the drawing.

Once all the cards were drawn, it was time to send them. This required addressing envelopes, and since plotting is so much fun the envelopes were also plotted. The envelopes took an additional 18 minuted each to plot on average, due mostly to plotting text being time consuming.

All of the code for this project is available on GitHub.

Adventures in Logging

2017-11-19T22:17:45-08:00

This post is the first in a series on using the logging facilities in Python. Logging is an incredibly powerful tool for developers during early system development, and also during deployment.

One of the interesting features of the logging package is an optimization where rendering of the log message is delayed as long as possible. What's not obvious is how this optimization will impact execution time.

In order to understand the consequences of how we construct the arguments to a call to a logger, we're going to run three experiments. The first experiment will explore logging performance with a single logger, the second will use two loggers, and the third will use three loggers.

Logging Configuration

In order to study only the effect of the arguments on logging performance, we use a custom logging.Handler subclass that performs no I/O, but does go through the motions of preparing the text for a log message.

    class FormattingNullHandler(logging.Handler):
        """Custom logging.Handler for performance measurements."""

        def emit(self, record):
            """Ensure that the record is formatted, but avoid doing I/O. This
            ensures that our expensive argument processing always occurs."""
            ignored_result = self.format(record)

Logging was configured as follows for the single handler experiment:

 1    # Configure logging
 2    logger = logging.getLogger("")
 3    logging._srcfile = None  # Disable file lookup.
 4    logging.logThreads = 0  # Disable logging thread information
 5    logging.logProcesses = 0  # Disable logging process information
 6    logging.logMultiprocessing = 0  # Disable logging multiprocess information
 7
 8    # Configure a single log handler with a minimalist formatter.
 9    handler = FormattingNullHandler()
10    formatter = logging.Formatter('%(message)s')
11    handler.setFormatter(formatter)
12    logger.addHandler(handler)

Experimental Code

 1    def exercise_driver():
 2        """Exercise the logging system, with different reporting levels."""
 3        results = {}
 4        for log_level in LOGGING_LEVELS:
 5            level = logging.getLevelName(log_level)
 6            logger.setLevel(log_level)
 7            results[level] = exercise_log()
 8        return results
 9
10
11    def exercise_log(iteration_limit=10, expensive_iterations=2500000):
12        """Run the experiment with a logger that has been externally configured."""
13        timer = Timer()
14        running_total = 0
15        iteration = 0
16        expensive_operation = ExpensiveCounter(expensive_iterations=expensive_iterations)
17
18        while iteration < iteration_limit:
19            with timer, expensive_operation:
20                for level in LOGGING_LEVELS:
21                    logger.log(level, "%s", expensive_operation)
22            running_total += timer.time()
23            iteration += 1
24
25        object_counter = expensive_operation.count
26        object_time = running_total / iteration_limit
27
28        running_total = 0
29        iteration = 0
30        while iteration < iteration_limit:
31            with timer, expensive_operation:
32                for level in LOGGING_LEVELS:
33                    logger.log(level, "%s", expensive_operation())
34            running_total += timer.time()
35            iteration += 1
36
37        invoked_counter = expensive_operation.count
38        invoked_time = running_total / iteration_limit
39
40        return {'invoked_count': invoked_counter,
41                'invoked_time': invoked_time,
42                'obj_count': object_counter,
43                'obj_time': object_time}

The experimental code makes 2 logging calls in exercise_log. The first call, on line 21, passes an expensive_operation object reference as an argument to the logging call, while the second call, on line 33, invokes the expensive_operation object, passing the result to the logging call. In both cases the logging call is made for each of the standard log levels. The exercise_log function is called for each of the standard log levels. The expensive_operation object is just a while loop that counts from 0 to expensive_iterations by 1s when it is invoked or converted to a string.

Tests

The initial test uses one log handler and has the following results:

Execution times in seconds for logging when passing an object vs a method invocation as log message parameter to a single logger with one handler.
Level Filter	Obj Ref	Avg. Time	Method Call	Avg. Time
CRITICAL	1	0.0434	5	0.2182
ERROR	2	0.0861	5	0.2177
WARNING	3	0.1294	5	0.2177
INFO	4	0.1724	5	0.2179
DEBUG	5	0.2427	5	0.2450

The first item of note is the Obj Ref column, which is the count of the number of times the object was converted to a string. In keeping with the behavior that message formatting is deferred as long as possible, we see that the string conversion only happened when the message would be rendered under the logging level configuration. When we passed the result of invoking the object, we performed the expensive operation every time, resulting in longer execution times.

The second test uses two handlers associated with a single logger and had the following results:

Execution times seconds for logging when passing an object vs a method invocation as log message parameter to a single logger with two handlers.
Level Filter	Obj Ref	Avg. Time	Method Call	Avg. Time
CRITICAL	2	0.0864	5	0.2175
ERROR	4	0.1732	5	0.2188
WARNING	6	0.2600	5	0.2202
INFO	8	0.3466	5	0.2186
DEBUG	10	0.4310	5	0.2181

The third test uses three handlers associated with a single logger and had the following results:

Execution times in seconds for logging when passing an object vs a method invocation as log message parameter to a single logger with three handlers.
Level Filter	Obj Ref	Avg. Time	Method Call	Avg. Time
CRITICAL	3	0.1295	5	0.2174
ERROR	6	0.2595	5	0.2174
WARNING	9	0.3890	5	0.2174
INFO	12	0.5198	5	0.2180
DEBUG	15	0.6494	5	0.2183

Looking at the second and third tests we see that the number of times we convert an object to a string increases with the number of handlers. The initial test might have given the impression that object references are better, but we can see that the situation is more complicated.

Discussion

As we can see from the test results, minimizing the amount of time we spend in logging calls is not simple. We need to be mindful of the number of handlers, the logging level, and the cost of preparing the arguments for the logger. In general it will be better to have the arguments created once for a logging call, avoiding the need to perform expensive operations for each handler.

The logging cookbook recommends using the logger.isEnabledFor(level) function to decide if a logging call should be made. This practice is unsatisfying as it encourages the use of expensive functions as arguments to logging calls. A better option would be to refactor your code so that the generation of arguments for logging calls is inexpensive, using the logger.isEnabledFor(level) method only when inexpensive arguments cannot be used.

An important caveat for logging calls, regardless of how computational expense is managed, is that they should not alter system state. If it is necessary to alter system state via functions that generate input for logging arguments, then those functions should be idempotent.

What Flow Control?

2016-09-29T14:44:12-07:00

The `assert()` macro is not a flow control mechanism

An interesting vulnerability in systemd appears to be a consequence of using the assert() macro as a flow control construct. In the code the authors had the following function:

 /* Extract from systemd/src/core/manager.c */
 void manager_invoke_notify_message(Manager *m, Unit *u, pid_t pid, const char *buf, size_t n, FDSet *fds) {
    _cleanup_strv_free_ char **tags = NULL;

    assert(m);
    assert(u);
    assert(buf);
    assert(n > 0);

    tags = strv_split(buf, "\n\r");
    if (!tags) {
        log_oom();
        return;
    }

    if (UNIT_VTABLE(u)->notify_message)
        UNIT_VTABLE(u)->notify_message(u, pid, tags, fds);
    else
        log_unit_debug(u, "Got notification message for unit. Ignoring.");
}

In line 8 of the snippet there is an assertion to check if n > 0, in this case n is the size of a buffer. Since a buffer of size 0 causes systemd to crash on some systems, this is a good check. There are however two serious problems with the code as written:

If systemd is compiled with the -DNDEBUG compiler flag, then the assert() macro will not generate any code so the check doesn't happen.
If systemd is compiled without the -DNDEBUG</span> flag, then if the assertion fails the program will abort.

For a critical software component, both problems are undesirable. The core of the problem appears to be the use of the assert() macro as a flow control mechanism. If it critically important for the value of n to be checked, then use a real comparison with explicit error handling. An additional item of concern is the use of assert() macros where there is an implicit assumption that NULL == 0 in lines 5 - 7. At a minimum, comparisons of a pointer to NULL should be done explicitly.

Assuming that the conditions implied by the assert() macro usage is correct, the code should be rewritten as:</p>

/* Revised code. */
static void manager_invoke_notify_message(Manager *m, Unit *u, pid_t pid, const char *buf, size_t n, FDSet *fds) {
    _cleanup_strv_free_ char **tags = NULL;

    assert(m != NULL);
    if (m == NULL){
        // log as appropriate
        return;
    }
    assert(u != NULL);
    if (u == NULL){
        // log as appropriate
        return;
    }
    assert(buf != NULL);
    if (buf == NULL){
        // log as appropriate
        return;
    }
    assert(n > 0);
    if (n <= 0){
        // log as appropriate
        return;
    }

    tags = strv_split(buf, "\n\r");
    if (!tags) {
        log_oom();
        return;
    }

    if (UNIT_VTABLE(u)->notify_message)
        UNIT_VTABLE(u)->notify_message(u, pid, tags, fds);
    else
        log_unit_debug(u, "Got notification message for unit. Ignoring.");
}

In keeping with good practice, this code will check important conditions regardless of the presence of the -DNDEBUG compiler flag. Additionally, in keeping with the C11 standard ISO/IEC 9899:201x NULL is not assumed to be a zero of any type.

Living with Exceptions

2014-05-11T21:39:17-07:00

In Java, exceptions occur when some non-planned issue occurs that interrupts the normal flow of execution. Exception handling is accomplished in one of two general ways. The first is to simply allow the exception to propagate up the call stack. This is accomplished by adding the throws keyword and a list of exception types to the method declaration. The second approach is to execute code that may throw an exception inside of a try block. Immediately following the try block will be one or more catch blocks that catch the different exception types that may occur in the preceding try block. The catch block(s) may be followed by a finally block that is used for cleanup.

In the code sample below, an exception is raised at line 3 triggering a change in the expected flow of control in the program. Since the exception in some_method() was not in a try block, control transfers back to a_method(), which was invoked at line 8. In a normal (non-exceptional) situation the program would advance to line 9. Since the exception is in a try block, execution will transfer to the associated catch block to handle the exception. In the catch block the exception may be handled, ignored, or re-thrown. Note that at line 12 there is a return statement. If this code was not being executed as part of exception handling, control would return to the main() method. However, since this code is part of exception handling and there is a finally block, the code at line 15 will be executed before the return takes effect. If line 12 was a System.exit() invocation, the finally block would still be executed prior to the exit occurring.

 public class ExceptionHandling {
     public static void some_method() throws Exception {
         throw new Exception("Manually Thrown Exception");
     }
     public static void a_method(){
         try {
             System.out.println("Before Exception");
             some_method();
             System.out.println("After Exception");
         } catch (Exception e) {
             System.out.println("Handling Exception");
            return;
         }
         finally{
             System.out.println("Entered Finally");
         }
     }
     public static void main(String[] args){
         System.out.println("Calling Method");
         a_method();
         System.out.println("Returned From Method");
     }
 }

Note in the output below that After Exception is never printed. Once an exception has occurred, the execution in the try block is done.

Calling Method
Before Exception
Handling Exception "Manually Thrown Exception"
Entered Finally
Returned From Method

A common pattern of execution is to handle the exception, and then continue execution after the try-catch-finally construct as shown below.

 public class ExceptionHandlingRecover {
     public static void some_method() throws Exception {
         throw new Exception("Manually Thrown Exception");
     }
     public static void a_method(){
         try {
             System.out.println("Before Exception");
             some_method();
             System.out.println("After Exception");
         } catch (Exception e) {
             System.out.println("Handling Exception");
         }
         finally{
             System.out.println("Entered Finally");
         }
         System.out.println("Continuing After try-catch-finally");
     }
     public static void main(String[] args){
         System.out.println("Calling Method");
         a_method();
         System.out.println("Returned From Method");
     }
 }

Calling Method
Before Exception
Handling Exception
Entered Finally
Continuing After try-catch-finally
Returned From Method

This is potentially dangerous since the execution context may not be what you expect, especially if more than one exception is possible in try block.

Exception Handling Strategies

There are three basic strategies for managing exceptions. The first strategy is to ignore them, the second is to address them, and the third is to make them someone else's problem. Each strategy has some benefits and costs, and it is the responsibility of the developer to choose the correct strategy.

Ignoring Exceptions

Simply ignoring the exception by having a catch block that does not change program state is a tempting strategy that is unfortunately supported by many IDEs via code generation wizards. A common form of ignoring exceptions is shown below:

try {
    some_exception_causing_method();
} catch (Exception e) {
    e.printStackTrace();
}

Cost

Ignoring exceptions is tempting since you can simply continue along as if nothing happened. In the code sample a stack trace would be printed to the error console, but there is no guarantee that anyone would ever see it. An exception may be an indication that system state is such that the program cannot continue execution producing good results. Failure to appropriately address an exception may result in undefined behavior that is extremely difficult to diagnose and correct since the diagnostic information is only available in the error console. In some execution environments that output is not preserved, rendering it useless for debugging.

Benefit

There may be cases where an exception simply does not matter, and may be safely ignored. This is rare since exceptions are an indication that something is not working as expected. If an exception is ignored, it is crucial to document why the exception was ignored.

Caveats

A common pattern for reading files is to use an infinite loop to read the file, relying on catching an IOException when the end of file is reached to exit the loop. While this works, it is extremely poor practice. Reaching the end of a file is not the only reason an IOException could be thrown. Additionally, using the exception handling mechanism to break loops can make tracing code execution difficult.

Addressing Exceptions

For applications, addressing exceptions is the common strategy. Addressing an exception may take the form of recovering from the exceptional situation, or it may be alerting the user that there is a problem and terminating the program.

Costs

Addressing an exception by recovering may be challenging to the point of being impractical.

Benefits

If you are able to recover from an exception then the program does not need to terminate, thus leading to a better user experience. In the case where recovery is not possible, capturing the problem early and terminating may make debugging easier and also reduce the likelihood of data corruption and spurious program behavior.

Caveats

While it is tempting to recover from exceptions, this is often not advisable. Unless the execution context's validity is independent of the exception or can be repaired despite the exception, it may be the case that the program cannot correctly execute after an exception. Continuing execution is a good goal, but should only be attempted in cases when terminating the program is not an option or the exception does not lead to an invalid program state.

Making Exceptions Someone Else's Problem

For the developers of libraries and web services, this is often the correct strategy. An exception may arise because of invalid input provided to a library function. It is inappropriate for a library to attempt to correct an exception, but should instead indicate that an exception has occurred. In the case of web services, exceptions may arise as a consequence of corrupted web requests. A viable strategy for web services is to raise an exception to inform the web service container that there was a problem and let the container alert the client in an appropriate manner.

Costs

When throwing exceptions it is crucial that the exceptions be meaningful and provide diagnostic information back to the calling methods so they can deal with the exception correctly.

Benefits

By making the exceptions the problem of the caller, the code raising the exceptions can be simpler and focus on operating on correct inputs.

Caveats

As with any time you cause problems for somebody else, be sure that is an appropriate action. Additionally, when you are writing an application, simply re-throwing exceptions will likely result in the application crashing. In Java (and other languages with exceptions) any exceptions that are not caught by the application will be caught by the runtime environment and may result in application termination without any cleanup.

Conclusions

Exception handling introduces a mechanism for altering the flow-of-control when issues arise during program execution. The proper response to exceptions depends on the context in which the exception occurs. Regardless of the context exceptions should be dealt with deliberately ensuring that they are not simply ignored.

Many other languages have similar exception methods. While the syntax varies, the issues are largely the same.

Bespoke Bytes

Biscuits

Making Biscuits

Basic Recipe

Variations

Spam & Cheese

Sweet Rockets

Holiday Cards 2022

Setting the Scene

Coding the Card

Adventures in Logging

Logging Configuration

Experimental Code

Tests

Discussion

What Flow Control?

The assert() macro is not a flow control mechanism

Living with Exceptions

Exception Handling Strategies

Ignoring Exceptions

Cost

Benefit

Caveats

Addressing Exceptions

Costs

Benefits

Caveats

Making Exceptions Someone Else's Problem

Costs

Benefits

Caveats

Conclusions

The `assert()` macro is not a flow control mechanism