A Library to Ease Accessing Flash-based (PROGMEM) Data

Storing static program data in flash/PROGMEM is a tricky part of Arduino programming. To save precious RAM, a novice user already at odds with unfamiliar C++ syntax must digest such daunting concepts as prog_char, PSTR(), PROGMEM, pgm_read_word(), etc. Even seasoned users get tripped up by the indirection and typecasting that are required to retrieve valid PROGMEM data. Add to that a couple of apparent bugs in the implementation, and it’s clear that PROGMEM is a complicated mess.

I have written a new library, Flash, which abstracts away most of this complexity. It provides new String, Array, Table, and String Array types that make ROM-based data collections as easy to use as “normal” types. Each overrides the C++ [] operator, so to extract individual elements one uses familiar array access syntax:

// float array example
float t = temperatures[950];

// (2D) font table example
digitalWrite(pin[i], font_table['a'][i]);

// string example
for (int i=0; i<str.length(); ++i)
  if (str[i] == ';') break;

Creating named Flash objects

To create a Flash object, you use a library-provided macro. Each of the four basic types provides its own creation macro:

Strings: FLASH_STRING(name, value)
Arrays: FLASH_ARRAY(type, name, list of values…)
Tables: FLASH_TABLE(type, name, columns, values…)
String Arrays: FLASH_STRING_ARRAY(name, values…)

Examples:

FLASH_STRING(big_string, "Moreover, as you might appreciate, "
    "their implications were such as to provoke a certain "
    "degree of sorrow within me. Indeed - why should I not "
    "admit it? - at that moment, my heart was breaking.");

FLASH_ARRAY(float, temperatures, 23.1, 23.1, 23.2, 23.2, 23.4,
    23.7, 25.0, 26.0, 26.8, 28.8, 30.2, 31.9, 33.1, 33.1, 33.2,
    33.2, 33.4, 33.7, 35.0, 36.0, 36.8, 38.8, 40.2, 41.9);

FLASH_TABLE(boolean, font_table, 7, {0,0,0,0,0,0,0},
    {0,0,0,1,0,1,1}, {1,0,1,1,1,0,1}, {1,1,0,0,0,0,0},
    {0,1,0,1,0,1,0}, {1,0,1,1,1,0,1}, {1,0,0,1,0,0,1},
    {0,0,1,1,0,1,1}, {1,0,1,1,1,1,1});

FLASH_STRING_ARRAY(digits, PSTR("Zero"), PSTR("One"),
    PSTR("Two"), PSTR("Three"), PSTR("Four"), PSTR("Five"),
    PSTR("Six"), PSTR("Seven"), PSTR("Eight"), PSTR("Nine"));

[Note that you can make Arrays and Tables out of any native type.]

[Note also that the String Array is a slightly special case. For technical reasons, as you can see in the example, each member of the array initializer must be wrapped with the PSTR() operator. For this reason, String Arrays cannot be declared at global scope, but must be non-statically defined within in the body of a function. Note also that while the individual strings in a String Array are contained in PROGMEM space, the array itself (again, for technical reasons) is RAM-based, consuming 2 bytes per array element. This limitation does not apply to the other Flash types.]

Using named Flash objects

Array-like access. Each Flash type provides an [] operator to get access to the underlying data, for example:

    // extract an element from an Array
    float t = temperatures[5];

    // examine characters in a String
    for (int i=0; i<big_string.length(); ++i)
      if (big_string[i] == 'O') ++big_O_counter;

    // Use a font table to configure a 7-segment display
    for (int i=0; i<font_table[1].count(); ++i)
      digitalWrite(pins[i], font_table[1][i]);

    // Examine the sizes of strings in a String Arra
    for (int i=0; i<digits.count(); ++i)
      Serial.println(digits[i].length());

Since Tables are really just two-dimensional arrays, the Table object’s implementation of [] returns a one-dimensional Array object.

Size. Each Flash type provides a mechanism for determining its size:

String and String Array: size_t length();
Array: size_t count();
Table: size_t rows(); size_t cols();

Access. If you need it, each Flash object provides an access() method that returns the underlying PROGMEM pointer

Print compatibility. Each Flash object has a print() method that can be used to serialize the object to any stream derived from core class Print (HardwareSerial, NewSoftSerial, LiquidCrystal, Ethernet, PString, etc.) See “Printing Flash objects” below.

String copy. Flash String objects may be copied in whole or part to RAM using the String object’s copy method:

    // copies the first 10 characters of the big_string
    char temp[11];
    big_string.copy(temp, 10);

Printing Flash objects

Each Flash object can “print” itself, using either its print method directly:

    temperatures.print(Serial);
    big_string.print(lcd);
    font_table.print(newsoftserial);

or, equivalently, by using the << streaming operator:

    Serial << temperatures;
    lcd << big_string;
    newsoftserial << font_table;

Flash objects can also be “printed” using the familiar Serial.print[ln] syntax, but this requires a small patch to the core Print.h file. See “Patching Print.h” below

Using unnamed (inline) Flash Strings

One of the most useful features of the Flash library is the capability to print unnamed strings embedded directly within an expression. In other words, if you want to use a ROM string in a print statement, it is not necessary to declare a string variable and print it like this:

    FLASH_STRING(big_string12,
      "'Twas brillig and the slithy toves did gyre and gimble");
    lcd << big_string12;

Instead you can simply use the new built-in F() macro (F for “Flash”) to do the same thing inline:

    lcd << F("'Twas brillig and the slithy toves did gyre and gimble");

And again, you can also use

    lcd.print(F("'Twas brillig and the slithy toves did gyre and gimble"));

to print an inline string, but this requires the patch to Print.h.

Patching Print.h (optional)

By making a very small change to the core Print.h file, you can add the capability to print either named or unnamed Flash objects using the familiar print[ln] syntax. What the patch does is tell class Print about Flash objects and how to print them. Note that while this patch is useful, it is entirely optional. Flash already provides two mechanisms for printing ROM strings and data structures:

    obj.print(stream);
    stream << obj;

This patch adds a third technique, the familar

    stream.print(obj);

To patch, add the following interface definition to the top of Print.h, right after the #include statements. This defines what a Flash object is, from the perspective of the Print class.

    #define ARDUINO_CORE_PRINTABLE_SUPPORT
    class Print;
    class _Printable
    {
    public:
      virtual void print(Print &stream) const = 0;
    };

Then add a couple of inline print methods — alongside all the others — to the body of class Print:

    void println(const _Printable &obj)
    { obj.print(*this); println(); }
    void print(const _Printable &obj)
    { obj.print(*this); }

No need to recompile anything; you are now ready to roll:

    Serial.print(myarray);
    Serial.print(mytable);
    Serial.print(F("A long, long, unnamed inline ROM string"));

Library Version

You can retrieve the version of the Flash library by inspecting FLASH_LIBRARY_VERSION.

int ver = FLASH_LIBRARY_VERSION;

Download

The latest version of Flash is available here: Flash3.zip

Change Log

1. initial version
2. moved some code to a new .cpp file to avoid inlining every member function
3. fixed a bug in the copy() method

Acknowledgements

Thanks to forum user Bill W. (”westfw”) for testing Flash and observing that Strings take up a surprising amount of extra PROGMEM space (20 bytes per string). I used his data to modify the library so that some of the commonly used functions, notably the _FLASH_STRING constructor are not inlined. This change reduces the footprint of the example programs noticeably. Thanks, Bill!

I appreciate any suggestions or input.

Mikal Hart