Flash

A Library to Ease Accessing Flash-based (PROGMEM) Data

Note: Flash 4.0 now supports Arduino 1.0!

Storing static program data in flash/PROGMEM is a tricky part of Arduino programming. To save precious RAM, a novice user already at odds with unfamiliar C++ syntax must digest such daunting concepts as prog_char, PSTR(), PROGMEM, pgm_read_word(), etc. Even seasoned users get tripped up by the indirection and typecasting that are required to retrieve valid PROGMEM data. Add to that a couple of apparent bugs in the implementation, and it’s clear that PROGMEM is a complicated mess.

I have written a new library, Flash, which abstracts away most of this complexity. It provides new String, Array, Table, and String Array types that make ROM-based data collections as easy to use as “normal” types. Each overrides the C++ [] operator, so to extract individual elements one uses familiar array access syntax:

// float array example
float t = temperatures[950];

// (2D) font table example
digitalWrite(pin[i], font_table['a'][i]);
// string example
for (int i=0; i<str.length(); ++i)
  if (str[i] == ';') break;

Creating named Flash objects

To create a Flash object, you use a library-provided macro. Each of the four basic types provides its own creation macro:

Strings: FLASH_STRING(name, value)
Arrays: FLASH_ARRAY(type, name, list of values…)
Tables: FLASH_TABLE(type, name, columns, values…)
String Arrays: FLASH_STRING_ARRAY(name, values…)

Examples:

FLASH_STRING(big_string, "Moreover, as you might appreciate, "
    "their implications were such as to provoke a certain "
    "degree of sorrow within me. Indeed - why should I not "
    "admit it? - at that moment, my heart was breaking.");
FLASH_ARRAY(float, temperatures, 23.1, 23.1, 23.2, 23.2, 23.4,
    23.7, 25.0, 26.0, 26.8, 28.8, 30.2, 31.9, 33.1, 33.1, 33.2,
    33.2, 33.4, 33.7, 35.0, 36.0, 36.8, 38.8, 40.2, 41.9);
FLASH_TABLE(boolean, font_table, 7, {0,0,0,0,0,0,0},
    {0,0,0,1,0,1,1}, {1,0,1,1,1,0,1}, {1,1,0,0,0,0,0},
    {0,1,0,1,0,1,0}, {1,0,1,1,1,0,1}, {1,0,0,1,0,0,1},
    {0,0,1,1,0,1,1}, {1,0,1,1,1,1,1});
FLASH_STRING_ARRAY(digits, PSTR("Zero"), PSTR("One"),
    PSTR("Two"), PSTR("Three"), PSTR("Four"), PSTR("Five"),
    PSTR("Six"), PSTR("Seven"), PSTR("Eight"), PSTR("Nine"));

[Note that you can make Arrays and Tables out of any native type.]

[Note also that the String Array is a slightly special case. For technical reasons, as you can see in the example, each member of the array initializer must be wrapped with the PSTR() operator. For this reason, String Arrays cannot be declared at global scope, but must be non-statically defined within in the body of a function. Note also that while the individual strings in a String Array are contained in PROGMEM space, the array itself (again, for technical reasons) is RAM-based, consuming 2 bytes per array element. This limitation does not apply to the other Flash types.]

Using named Flash objects

Array-like access. Each Flash type provides an [] operator to get access to the underlying data, for example:

    // extract an element from an Array
    float t = temperatures[5];
    // examine characters in a String
    for (int i=0; i<big_string.length(); ++i)
      if (big_string[i] == 'O') ++big_O_counter;
    // Use a font table to configure a 7-segment display
    for (int i=0; i<font_table[1].count(); ++i)
      digitalWrite(pins[i], font_table[1][i]);
    // Examine the sizes of strings in a String Arra
    for (int i=0; i<digits.count(); ++i)
      Serial.println(digits[i].length());

Since Tables are really just two-dimensional arrays, the Table object’s implementation of [] returns a one-dimensional Array object.

Size. Each Flash type provides a mechanism for determining its size:

String and String Array: size_t length();
Array: size_t count();
Table: size_t rows(); size_t cols();

Access. If you need it, each Flash object provides an access() method that returns the underlying PROGMEM pointer

Print compatibility. Each Flash object has a print() method that can be used to serialize the object to any stream derived from core class Print (HardwareSerial, NewSoftSerial, LiquidCrystal, Ethernet, PString, etc.) See “Printing Flash objects” below.

String copy. Flash String objects may be copied in whole or part to RAM using the String object’s copy method:

    // copies the first 10 characters of the big_string
    char temp[11];
    big_string.copy(temp, 10);

Printing Flash objects

Each Flash object can “print” itself, using either its print method directly:

    temperatures.print(Serial);
    big_string.print(lcd);
    font_table.print(newsoftserial);

or, equivalently, by using the << streaming operator:

    Serial << temperatures;
    lcd << big_string;
    newsoftserial << font_table;

Flash objects can also be “printed” using the familiar Serial.print[ln] syntax, but this requires a small patch to the core Print.h file. See “Patching Print.h” below

Library Version

You can retrieve the version of the Flash library by inspecting FLASH_LIBRARY_VERSION.

int ver = FLASH_LIBRARY_VERSION;

Download

The latest version of Flash is available here: Flash4.zip

Change Log

  1. initial version
  2. moved some code to a new .cpp file to avoid inlining every member function
  3. fixed a bug in the copy() method

Acknowledgements

Thanks to forum user Bill W. (“westfw”) for testing Flash and observing that Strings take up a surprising amount of extra PROGMEM space (20 bytes per string). I used his data to modify the library so that some of the commonly used functions, notably the _FLASH_STRING constructor are not inlined. This change reduces the footprint of the example programs noticeably. Thanks, Bill!

I appreciate any suggestions or input.

Mikal Hart

Page last updated on January 28, 2012 at 12:16 am
131 Responses → “Flash”

  1. Mikal

    3 months ago

    @David,

    I suspect that a 64K byte table is too big of a single structure, since its size cannot be encoded in 16 bits. Can you reorganize it into 2 distinct arrays?


  2. Mikal

    3 months ago

    @Rick,

    I think I’d make a simple loop, something like:

    void setup()
    {
    Serial.begin(115200);
    for (int i=0; i<tempvoltage.count(); ++i)
    if (tempvoltage[i] == 1.5)
    Serial.println(temperatures[i]);
    }

    void loop(){}


  3. Mikal

    3 months ago

    @dulises, Flash now works with Arduino 1.0. Thank you. M


  4. Mikal

    3 months ago

    Geez, this is a mystery! I don’t know. Anyone?


  5. Franz Wagner

    3 months ago

    Mikal,
    thank you for updating Flash 4 to Arduino 1.0!
    With EthernetClient client:
    client.print(7,DEC);
    client << _DEC(7);
    last one takes 4 bytes more of extra PROGMEM space.
    But not in use with Serial << _DEC(7);
    regards
    Franz Wagner


  6. Nick

    3 months ago

    Does this library allow Arrays to be sent and stored to the arduino?
    I want to be able to send {1000,900,800,700,600,500,400,300,200,100,10,9,8,7,6,5,4,3,2,1,0}
    and save it as Array1.
    i.e. unsigned int Array1 = {1000,900,800,700,600,500,400,300,200,100,10,9,8,7,6,5,4,3,2,1,0};
    Then I could use the command delay(Array1[i]);
    I would send the array over the serial port through a program called termite.exe.
    I would need the array to be saved on the arduino not sent one element at a time.


  7. Mikal

    3 months ago

    @Franz,

    Hmm… Thanks for sharing, but I can’t see why.


  8. Mikal

    3 months ago

    @Nick,

    Sorry, the PROGMEM (Flash) data region can only be populated at program burn time. You can’t change the values in the flash after the program has started running.


  9. francis

    2 months ago

    hello
    if i use standard
    int firstarray[] = {n,…..,n};
    int secondarray[] = {n,…..,n};
    these instructions works:
    code[0] = fistarray[i];
    code[1] = secondarray[i];
    but using
    FLASH_ARRAY(int, firstarray,n,…….,n);
    FLASH_ARRAY(int, secondarray,n,…….,n);
    code[0] = fistarray[i];
    code[1] = secondarray[i]; this return the value in firstarray not the one in secondarray
    why ?
    thanks


  10. Rajarajan Rajamani

    2 months ago

    Started compiling the Flash library files (in FreeBSD using gmake) using arduino-1.0 headers/etc. Compilation fails for Flash.cpp/Flash.h in the places where prog_char is used. As noted in https://github.com/arduino/Arduino/commit/0acebeeff48cdb28c602f04e1a5906c6b2149a02 I changed all occurances of prog_char to char PROGMEM and it compiled.


  11. Ron

    2 months ago

    /*I asked this question on the arduino board, but it occurs to me that I should just come to the source, so here I am*/

    Hi, I am pretty limited in my programming knowledge and today figured out that the reason my long-time project stopped working is because I have run out of sram. Looks like I need to put my many arrays into flash/PROGMEM. PROGMEM documentation scares me, and your flash library looks much easier. Still, I am having trouble with the new syntax I need to adjust my code.

    My program used to have arrays that looked like:

    int arrayOne[] = //recently made into bytes,
    24, 26, 28, 29, 31, 33, 35, //actually
    36, 38, 40, 41, 43, 45, 47,
    48, 50, 52, 53, 55, 57, 59,
    60, 62, 64, 65, 67, 69, 71,
    72, 74, 76, 77, 79, 81, 83,
    84, 86, 88, 89, 91, 93, 95,
    96, 98, 100, 101, 103, 105, 107,
    };

    … which I later called thusly:

    variable=(mappedVariable[arrayOne]);

    Using the flash library, I changed the array to:

    FLASH_ARRAY(int, arrayOne,
    24, 26, 28, 29, 31, 33, 35,
    36, 38, 40, 41, 43, 45, 47,
    48, 50, 52, 53, 55, 57, 59,
    60, 62, 64, 65, 67, 69, 71,
    72, 74, 76, 77, 79, 81, 83,
    84, 86, 88, 89, 91, 93, 95,
    96, 98, 100, 101, 103, 105, 107,
    );

    … but now I can’t figure out how to access the array. If I do

    variable=(mappedVariable[arrayOne]);

    the IDE won’t compile, saying:

    no match for ‘operator[] in mappedVariable[arrayOne]‘

    I tried to understand using the example you graciously provide but am too ignorant to move forward. Can you point me in the right direction?

    Thank you.


  12. Rajarajan Rajamani

    2 months ago

    Unable to compile the flash_streaming example.
    I am compiling on FreeBSD and was able to compile the flash_example code. However and trying to compile the flash_streaming example code, I am getting repeated compile errors for wherever the F() macro is used in the code
    like -

    avr-gcc flash_streaming.cpp -mmcu=atmega328p -DF_CPU=16000000UL -Os -w -Wl,–gc-sections -ffunction-sections -fdata-sections -DARDUINO=100 -I /home/rr/code/arduino//include/arduino/ -c -o flash_streaming.o
    flash_streaming.cpp: In function ‘void setup()’:
    flash_streaming.cpp:210:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:211:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:214:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:217:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:220:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:221:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:228:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    Any help would be appreciated.


  13. Ehsan

    2 months ago

    Hi,
    I get the following error when I compile FLASH_STRING_ARRAY
    statement-expressions are not allowed outside functions nor in template-argument lists

    what should I do?


  14. Mikal

    1 month ago

    Thanks for the tip, Raj.


  15. Mikal

    1 month ago

    Ron,

    It’s a peculiarity of C++ that array[index] and index[array] mean the same thing. But this peculiarity doesn’t carry over for FLASH_ARRAY objects. What you really want (in both old and new) is

    variable = arrayOne[mappedVariable];

    Mikal


  16. Paul S

    1 month ago

    I don’t know how to thank you for your code, but I was at my wits end trying to save memory on my OSEPP UNO. I had less than 100 bytes, things didn’t work, I knew I was running out of stack space. I had long lists of Strings to be used for input command processing… after I encoded the strings using your code.. I ended up with 582 bytes now free… more than ample to run reliably. Probably now can recover another 40 or so after some restructuring. My string comparison had to be done “badly” though. I had to do

    FLASH_STRING_ARRAY(vectCmd,PSTR( “ABORT” ),…. );

    char temp[30];
    vectCmd[ count ].copy(temp);

    if ( source == temp ) { …. }

    Didn’t like having to do that copy, but if you have a suggestion.. But thanks again!


  17. Mikal

    1 month ago

    Hmm…

    Try this addition at the top of flash_streaming.cpp, Raj:

    #undef F
    #define F(string_literal) (const_cast<__FlashStringHelper *>\
      (reinterpret_cast<const __FlashStringHelper *>(PSTR(string_literal))))
    

    Apparently some flavors of GCC don’t like reinterpret_cast removing const.


  18. EbiDK

    1 month ago

    How would I dynamically allocate some memory for an FLASH_STRING_ARRAY so I can use it from multiple functions?


  19. Andreas Sexauer

    1 month ago

    Hello,
    great library, but I think I found a bug. If you have a FLASH_ARRAY (tested with float type) and multiple var1 = flashArrayVar[pos1]; var2 = flashArrayVar[pos2]; var3 = flashArrayVar[pos3]; without any code in between in your program var1, var2 and var3 all have the same value. It is the value of flashArrayVar[pos1]. If I include an additional line of code between the lines with an delay(1); everything works as expected.
    var1 = flashArrayVar[pos1]; delay(1); var2 = flashArrayVar[pos2]; delay(1); var3 = flashArrayVar[pos3];
    It is even the same behavior with consecutive access to different FLASH_ARRAY Variables.
    Code showing the error: http://dl.dropbox.com/u/828982/flasharrayerror/flash_error.ino
    Output of this code: http://dl.dropbox.com/u/828982/flasharrayerror/output.txt
    Please feel free to contact me related this error.


  20. Mikal

    1 month ago

    @Ehsan,

    Can you give a more clear example of what you are talking about? I know that you can’t use PSTR, and by extension, F(), outside of a function body, but there isn’t any such restriction on FLASH_STRING_ARRAY.

    Mikal


  21. Mikal

    1 month ago

    @Paul S,

    Thanks for the kind comments, and yes! There is a “good” way to do want you want, namely:

    if (strcmp_P(source, vectCmd[count].access())) { …. }

    Mikal


  22. Mikal

    1 month ago

    @Andreas Sexauer,

    Thanks. This bizarre behavior has been observed elsewhere. Therefore, I’m now offering a $10 Reverse Geocache shield PCB to the first person who can explain why my fairly straightforward flash access code should be time sensitive! Anyone? You’ll also get everlasting fame!


  23. Pablo

    3 weeks ago

    @ Mikal

    Really good library. I use it to keep and send html ( server << F("HTTP/1.0…"); or server.print(F("HTTP/1.0…")); ) and it works Ok but the communication is very slow versus "server.print("HTTP/1.0…")". when i analised the HTML frame with wireshark i saw that for each byte of the array the F function sends one HTML message, i think this is because the function "F" send byte by byte to the print function the array. = server.print("H"); server.print("T"); server.print("T");…etc…
    Could be?
    Any idea how to avoid this?

    @Andreas Sexauer

    Time sensitive problem: on atmega 1281

    broken too _______________
    100
    100
    100
    100
    100
    100
    100
    100
    100
    100
    Working____________________
    100
    100
    100
    100
    100
    100
    11
    11
    11
    11
    working too _______________
    100
    11
    100
    11
    100
    11
    100
    11
    100
    11
    broken _______________
    100


  24. Mikal

    1 week ago

    @Pablo,

    Very interesting point and observation. Anyone have any suggestions how one might rectify the problem? Let me think on it…

    Mikal


  25. Colin

    1 week ago

    Hi Mikal, I have been using a class/library in Arduino to manage all my menus for a 2 line lcd program, during construction of the object I am passing it a pointer to the string:

    Menu Item1(“LINE 1 SETTINGS”);
    Menu Item11(“TIME”);
    Menu Item12(“PRESSURE”);

    However I ended up getting corrupted SRAM and am now looking into storing all the strings in flash instead. Do you have any examples of using your Flash library which sends a pointer of Progmem to a class object?


  26. Mikal

    1 week ago

    Hi Colin,

    The object created by the FLASH_STRING macro is of type _FLASH_STRING, so you can do

    FLASH_STRING(m1, “LINE 1 SETTINGS”);
    Menu item1(m1);

    Of course then you’d have to define the appropriate constructor for Menu. In this case it would be

    Menu(_FLASH_STRING &item)
    {…}


  27. Colin

    2 days ago

    Thanks for the example. I am getting some errors that i hope you can help me with. They are in regard to including ‘flash.h’ in the library header and .cpp file. The errors from arduino are:
    arduino-1.0\libraries\Flash/Flash.h:52: error: redefinition of ‘class _Printable’
    arduino-1.0\libraries\Flash/Flash.h:53: error: previous definition of ‘class _Printable’
    arduino-1.0\libraries\Flash/Flash.h:65: error: redefinition of ‘class _FLASH_STRING’
    arduino-1.0\libraries\Flash/Flash.h:66: error: previous definition of ‘class _FLASH_STRING’
    etc

4 Trackbacks For This Post
  1. New Flash library « Arduiniana

    [...] Interested to learn more? Read all about it here! [...]

  2. vkick Blog Site » Blog Archive » How to do Timer and Save Settings?

    [...] http://arduiniana.org/libraries/Flash/ Categories: Arduino Posted By: Mon Last Edit: 18 Jun 2009 @ 04 01 PM Email • Permalink Previous  Responses to this post » (None)  Post a Comment Click here to cancel reply. [...]

  3. Arduino net connected clock tells weather not time

    [...] third party libraries were used; PString to handle parsing the incoming website, Flash for memory management, and Streaming for easier coding. [...]

  4. Blog What I Made » YAHMS: Base Station

    [...] interface, see the links below for that too. You’ll also need NewSoftSerial of course and the Flash library which I’ve used to decrease memory usage. Follow the instructions in [...]

Leave a Reply