Flash

A Library to Ease Accessing Flash-based (PROGMEM) Data

Storing static program data in flash/PROGMEM is a tricky part of Arduino programming. To save precious RAM, a novice user already at odds with unfamiliar C++ syntax must digest such daunting concepts as prog_char, PSTR(), PROGMEM, pgm_read_word(), etc. Even seasoned users get tripped up by the indirection and typecasting that are required to retrieve valid PROGMEM data. Add to that a couple of apparent bugs in the implementation, and it’s clear that PROGMEM is a complicated mess.

I have written a new library, Flash, which abstracts away most of this complexity. It provides new String, Array, Table, and String Array types that make ROM-based data collections as easy to use as “normal” types. Each overrides the C++ [] operator, so to extract individual elements one uses familiar array access syntax:

// float array example
float t = temperatures[950];

// (2D) font table example
digitalWrite(pin[i], font_table['a'][i]);
// string example
for (int i=0; i<str.length(); ++i)
  if (str[i] == ';') break;

Download and Installation

To install this library, download here, unzip the archive into the Arduino “libraries” folder, and restart Arduino. You should rename the folder “Flash”, because the IDE doesn’t like dashes in the name.
Download

Creating named Flash objects

To create a Flash object, you use a library-provided macro. Each of the four basic types provides its own creation macro:

Strings: FLASH_STRING(name, value)
Arrays: FLASH_ARRAY(type, name, list of values…)
Tables: FLASH_TABLE(type, name, columns, values…)
String Arrays: FLASH_STRING_ARRAY(name, values…)

Examples:

FLASH_STRING(big_string, "Moreover, as you might appreciate, "
    "their implications were such as to provoke a certain "
    "degree of sorrow within me. Indeed - why should I not "
    "admit it? - at that moment, my heart was breaking.");
FLASH_ARRAY(float, temperatures, 23.1, 23.1, 23.2, 23.2, 23.4,
    23.7, 25.0, 26.0, 26.8, 28.8, 30.2, 31.9, 33.1, 33.1, 33.2,
    33.2, 33.4, 33.7, 35.0, 36.0, 36.8, 38.8, 40.2, 41.9);
FLASH_TABLE(boolean, font_table, 7, {0,0,0,0,0,0,0},
    {0,0,0,1,0,1,1}, {1,0,1,1,1,0,1}, {1,1,0,0,0,0,0},
    {0,1,0,1,0,1,0}, {1,0,1,1,1,0,1}, {1,0,0,1,0,0,1},
    {0,0,1,1,0,1,1}, {1,0,1,1,1,1,1});
FLASH_STRING_ARRAY(digits, PSTR("Zero"), PSTR("One"),
    PSTR("Two"), PSTR("Three"), PSTR("Four"), PSTR("Five"),
    PSTR("Six"), PSTR("Seven"), PSTR("Eight"), PSTR("Nine"));

[Note that you can make Arrays and Tables out of any native type.]

[Note also that the String Array is a slightly special case. For technical reasons, as you can see in the example, each member of the array initializer must be wrapped with the PSTR() operator. For this reason, String Arrays cannot be declared at global scope, but must be non-statically defined within in the body of a function. Note also that while the individual strings in a String Array are contained in PROGMEM space, the array itself (again, for technical reasons) is RAM-based, consuming 2 bytes per array element. This limitation does not apply to the other Flash types.]

Using named Flash objects

Array-like access. Each Flash type provides an [] operator to get access to the underlying data, for example:

    // extract an element from an Array
    float t = temperatures[5];
    // examine characters in a String
    for (int i=0; i      if (big_string[i] == 'O') ++big_O_counter;
    // Use a font table to configure a 7-segment display
    for (int i=0; i      digitalWrite(pins[i], font_table[1][i]);
    // Examine the sizes of strings in a String Array
    for (int i=0; i      Serial.println(digits[i].length());

Since Tables are really just two-dimensional arrays, the Table object’s implementation of [] returns a one-dimensional Array object.

Size. Each Flash type provides a mechanism for determining its size:

String and String Array: size_t length();
Array: size_t count();
Table: size_t rows(); size_t cols();

Access. If you need it, each Flash object provides an access() method that returns the underlying PROGMEM pointer

Print compatibility. Each Flash object has a print() method that can be used to serialize the object to any stream derived from core class Print (HardwareSerial, NewSoftSerial, LiquidCrystal, Ethernet, PString, etc.) See “Printing Flash objects” below.

String copy. Flash String objects may be copied in whole or part to RAM using the String object’s copy method:

    // copies the first 10 characters of the big_string
    char temp[11];
    big_string.copy(temp, 10);

Printing Flash objects

Each Flash object can “print” itself, using either its print method directly:

    temperatures.print(Serial);
    big_string.print(lcd);
    font_table.print(softserial);

or, equivalently, by using the << streaming operator:

    Serial << temperatures;
    lcd << big_string;
    softserial << font_table;

Library Version

You can retrieve the version of the Flash library by inspecting FLASH_LIBRARY_VERSION.

int ver = FLASH_LIBRARY_VERSION;

Download

The latest version of Flash is available here: Flash5.zip

Change Log

  1. initial version
  2. moved some code to a new .cpp file to avoid inlining every member function
  3. fixed a bug in the copy() method

Acknowledgements

Thanks to forum user Bill W. (“westfw”) for testing Flash and observing that Strings take up a surprising amount of extra PROGMEM space (20 bytes per string). I used his data to modify the library so that some of the commonly used functions, notably the _FLASH_STRING constructor are not inlined. This change reduces the footprint of the example programs noticeably. Thanks, Bill!

I appreciate any suggestions or input.

Mikal Hart

Page last updated on January 30, 2014 at 9:41 pm
252 Responses → “Flash”

  1. Erik

    12 years ago

    FYI: I posted a few other examples (with varying results) over at arduino.cc :
    http://arduino.cc/forum/index.php/topic,86019.0.html


  2. Peter

    12 years ago

    Hi Mikal

    really great job and many thanks for it. It’s the way it should be done using the compiler scopes. My sketch with a looooonnnngggg string showing a help text worked perfect.

    Many thanks again !!!


  3. Sorky

    12 years ago

    Your “unnamed (inline) Flash Strings” rock!

    Thanks Dude!


  4. josx

    12 years ago

    Hey
    Just you to know for using it with Arduino 1.0 Libraries you need to replace
    in line 22 of Flash.h Wprogram.h with Arduino.h


  5. Mikal

    12 years ago

    @David,

    I suspect that a 64K byte table is too big of a single structure, since its size cannot be encoded in 16 bits. Can you reorganize it into 2 distinct arrays?


  6. Mikal

    12 years ago

    @Rick,

    I think I’d make a simple loop, something like:

    void setup()
    {
    Serial.begin(115200);
    for (int i=0; i<tempvoltage.count(); ++i)
    if (tempvoltage[i] == 1.5)
    Serial.println(temperatures[i]);
    }

    void loop(){}


  7. Mikal

    12 years ago

    @dulises, Flash now works with Arduino 1.0. Thank you. M


  8. Mikal

    12 years ago

    Geez, this is a mystery! I don’t know. Anyone?


  9. Franz Wagner

    12 years ago

    Mikal,
    thank you for updating Flash 4 to Arduino 1.0!
    With EthernetClient client:
    client.print(7,DEC);
    client << _DEC(7);
    last one takes 4 bytes more of extra PROGMEM space.
    But not in use with Serial << _DEC(7);
    regards
    Franz Wagner


  10. Nick

    12 years ago

    Does this library allow Arrays to be sent and stored to the arduino?
    I want to be able to send {1000,900,800,700,600,500,400,300,200,100,10,9,8,7,6,5,4,3,2,1,0}
    and save it as Array1.
    i.e. unsigned int Array1 = {1000,900,800,700,600,500,400,300,200,100,10,9,8,7,6,5,4,3,2,1,0};
    Then I could use the command delay(Array1[i]);
    I would send the array over the serial port through a program called termite.exe.
    I would need the array to be saved on the arduino not sent one element at a time.


  11. Mikal

    12 years ago

    @Franz,

    Hmm… Thanks for sharing, but I can’t see why.


  12. Mikal

    12 years ago

    @Nick,

    Sorry, the PROGMEM (Flash) data region can only be populated at program burn time. You can’t change the values in the flash after the program has started running.


  13. francis

    12 years ago

    hello
    if i use standard
    int firstarray[] = {n,…..,n};
    int secondarray[] = {n,…..,n};
    these instructions works:
    code[0] = fistarray[i];
    code[1] = secondarray[i];
    but using
    FLASH_ARRAY(int, firstarray,n,…….,n);
    FLASH_ARRAY(int, secondarray,n,…….,n);
    code[0] = fistarray[i];
    code[1] = secondarray[i]; this return the value in firstarray not the one in secondarray
    why ?
    thanks


  14. Rajarajan Rajamani

    12 years ago

    Started compiling the Flash library files (in FreeBSD using gmake) using arduino-1.0 headers/etc. Compilation fails for Flash.cpp/Flash.h in the places where prog_char is used. As noted in https://github.com/arduino/Arduino/commit/0acebeeff48cdb28c602f04e1a5906c6b2149a02 I changed all occurances of prog_char to char PROGMEM and it compiled.


  15. Ron

    12 years ago

    /*I asked this question on the arduino board, but it occurs to me that I should just come to the source, so here I am*/

    Hi, I am pretty limited in my programming knowledge and today figured out that the reason my long-time project stopped working is because I have run out of sram. Looks like I need to put my many arrays into flash/PROGMEM. PROGMEM documentation scares me, and your flash library looks much easier. Still, I am having trouble with the new syntax I need to adjust my code.

    My program used to have arrays that looked like:

    int arrayOne[] = //recently made into bytes,
    24, 26, 28, 29, 31, 33, 35, //actually
    36, 38, 40, 41, 43, 45, 47,
    48, 50, 52, 53, 55, 57, 59,
    60, 62, 64, 65, 67, 69, 71,
    72, 74, 76, 77, 79, 81, 83,
    84, 86, 88, 89, 91, 93, 95,
    96, 98, 100, 101, 103, 105, 107,
    };

    … which I later called thusly:

    variable=(mappedVariable[arrayOne]);

    Using the flash library, I changed the array to:

    FLASH_ARRAY(int, arrayOne,
    24, 26, 28, 29, 31, 33, 35,
    36, 38, 40, 41, 43, 45, 47,
    48, 50, 52, 53, 55, 57, 59,
    60, 62, 64, 65, 67, 69, 71,
    72, 74, 76, 77, 79, 81, 83,
    84, 86, 88, 89, 91, 93, 95,
    96, 98, 100, 101, 103, 105, 107,
    );

    … but now I can’t figure out how to access the array. If I do

    variable=(mappedVariable[arrayOne]);

    the IDE won’t compile, saying:

    no match for ‘operator[] in mappedVariable[arrayOne]’

    I tried to understand using the example you graciously provide but am too ignorant to move forward. Can you point me in the right direction?

    Thank you.


  16. Rajarajan Rajamani

    12 years ago

    Unable to compile the flash_streaming example.
    I am compiling on FreeBSD and was able to compile the flash_example code. However and trying to compile the flash_streaming example code, I am getting repeated compile errors for wherever the F() macro is used in the code
    like –

    avr-gcc flash_streaming.cpp -mmcu=atmega328p -DF_CPU=16000000UL -Os -w -Wl,–gc-sections -ffunction-sections -fdata-sections -DARDUINO=100 -I /home/rr/code/arduino//include/arduino/ -c -o flash_streaming.o
    flash_streaming.cpp: In function ‘void setup()’:
    flash_streaming.cpp:210:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:211:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:214:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:217:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:220:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:221:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    flash_streaming.cpp:228:13: error: reinterpret_cast from type ‘const char*’ to type ‘__FlashStringHelper*’ casts away qualifiers
    Any help would be appreciated.


  17. Ehsan

    12 years ago

    Hi,
    I get the following error when I compile FLASH_STRING_ARRAY
    statement-expressions are not allowed outside functions nor in template-argument lists

    what should I do?


  18. Mikal

    12 years ago

    Thanks for the tip, Raj.


  19. Mikal

    12 years ago

    Ron,

    It’s a peculiarity of C++ that array[index] and index[array] mean the same thing. But this peculiarity doesn’t carry over for FLASH_ARRAY objects. What you really want (in both old and new) is

    variable = arrayOne[mappedVariable];

    Mikal


  20. Paul S

    12 years ago

    I don’t know how to thank you for your code, but I was at my wits end trying to save memory on my OSEPP UNO. I had less than 100 bytes, things didn’t work, I knew I was running out of stack space. I had long lists of Strings to be used for input command processing… after I encoded the strings using your code.. I ended up with 582 bytes now free… more than ample to run reliably. Probably now can recover another 40 or so after some restructuring. My string comparison had to be done “badly” though. I had to do

    FLASH_STRING_ARRAY(vectCmd,PSTR( “ABORT” ),…. );

    char temp[30];
    vectCmd[ count ].copy(temp);

    if ( source == temp ) { …. }

    Didn’t like having to do that copy, but if you have a suggestion.. But thanks again!


  21. Mikal

    12 years ago

    Hmm…

    Try this addition at the top of flash_streaming.cpp, Raj:

    #undef F
    #define F(string_literal) (const_cast<__FlashStringHelper *>\
      (reinterpret_cast<const __FlashStringHelper *>(PSTR(string_literal))))
    

    Apparently some flavors of GCC don’t like reinterpret_cast removing const.


  22. EbiDK

    12 years ago

    How would I dynamically allocate some memory for an FLASH_STRING_ARRAY so I can use it from multiple functions?


  23. Andreas Sexauer

    12 years ago

    Hello,
    great library, but I think I found a bug. If you have a FLASH_ARRAY (tested with float type) and multiple var1 = flashArrayVar[pos1]; var2 = flashArrayVar[pos2]; var3 = flashArrayVar[pos3]; without any code in between in your program var1, var2 and var3 all have the same value. It is the value of flashArrayVar[pos1]. If I include an additional line of code between the lines with an delay(1); everything works as expected.
    var1 = flashArrayVar[pos1]; delay(1); var2 = flashArrayVar[pos2]; delay(1); var3 = flashArrayVar[pos3];
    It is even the same behavior with consecutive access to different FLASH_ARRAY Variables.
    Code showing the error: http://dl.dropbox.com/u/828982/flasharrayerror/flash_error.ino
    Output of this code: http://dl.dropbox.com/u/828982/flasharrayerror/output.txt
    Please feel free to contact me related this error.


  24. Mikal

    12 years ago

    @Ehsan,

    Can you give a more clear example of what you are talking about? I know that you can’t use PSTR, and by extension, F(), outside of a function body, but there isn’t any such restriction on FLASH_STRING_ARRAY.

    Mikal


  25. Mikal

    12 years ago

    @Paul S,

    Thanks for the kind comments, and yes! There is a “good” way to do want you want, namely:

    if (strcmp_P(source, vectCmd[count].access())) { …. }

    Mikal


  26. Mikal

    12 years ago

    @Andreas Sexauer,

    Thanks. This bizarre behavior has been observed elsewhere. Therefore, I’m now offering a $10 Reverse Geocache shield PCB to the first person who can explain why my fairly straightforward flash access code should be time sensitive! Anyone? You’ll also get everlasting fame!


  27. Pablo

    12 years ago

    @ Mikal

    Really good library. I use it to keep and send html ( server << F("HTTP/1.0…"); or server.print(F("HTTP/1.0…")); ) and it works Ok but the communication is very slow versus "server.print("HTTP/1.0…")". when i analised the HTML frame with wireshark i saw that for each byte of the array the F function sends one HTML message, i think this is because the function "F" send byte by byte to the print function the array. = server.print("H"); server.print("T"); server.print("T");…etc…
    Could be?
    Any idea how to avoid this?

    @Andreas Sexauer

    Time sensitive problem: on atmega 1281

    broken too _______________
    100
    100
    100
    100
    100
    100
    100
    100
    100
    100
    Working____________________
    100
    100
    100
    100
    100
    100
    11
    11
    11
    11
    working too _______________
    100
    11
    100
    11
    100
    11
    100
    11
    100
    11
    broken _______________
    100


  28. Mikal

    12 years ago

    @Pablo,

    Very interesting point and observation. Anyone have any suggestions how one might rectify the problem? Let me think on it…

    Mikal


  29. Colin

    12 years ago

    Hi Mikal, I have been using a class/library in Arduino to manage all my menus for a 2 line lcd program, during construction of the object I am passing it a pointer to the string:

    Menu Item1(“LINE 1 SETTINGS”);
    Menu Item11(“TIME”);
    Menu Item12(“PRESSURE”);

    However I ended up getting corrupted SRAM and am now looking into storing all the strings in flash instead. Do you have any examples of using your Flash library which sends a pointer of Progmem to a class object?


  30. Mikal

    12 years ago

    Hi Colin,

    The object created by the FLASH_STRING macro is of type _FLASH_STRING, so you can do

    FLASH_STRING(m1, “LINE 1 SETTINGS”);
    Menu item1(m1);

    Of course then you’d have to define the appropriate constructor for Menu. In this case it would be

    Menu(_FLASH_STRING &item)
    {…}


  31. Reve

    12 years ago

    Dude, I just want to thank you for writing this. I’ve been banging my head against stupid glitches due to indirection weirdness for the better part of a week. With your library everything just _works_ out of the box.

    So again, thank you!

    – RM


  32. Colin

    12 years ago

    Thanks for the example. I am getting some errors that i hope you can help me with. They are in regard to including ‘flash.h’ in the library header and .cpp file. The errors from arduino are:
    arduino-1.0\libraries\Flash/Flash.h:52: error: redefinition of ‘class _Printable’
    arduino-1.0\libraries\Flash/Flash.h:53: error: previous definition of ‘class _Printable’
    arduino-1.0\libraries\Flash/Flash.h:65: error: redefinition of ‘class _FLASH_STRING’
    arduino-1.0\libraries\Flash/Flash.h:66: error: previous definition of ‘class _FLASH_STRING’
    etc


  33. Benni

    12 years ago

    Hi there! Thank you for this excellent work, it makes things so much easier!

    To me, there’s still an open question: can these Flash objects be modified when stored in RAM? Specifically, I need to read light intensities on Arduino’s analog inputs A0-A3, providing an array such as this:
    analogValues[4] = {A0, A1, A2, A3};
    I would like to store this array to a table in RAM for each iteration step (either in void loop() or a for-loop) but I can’t seem to overcome the issue of accessing the table (previously creates with
    FLASH_TABLE(int, input_values,4); ).


  34. Mikal

    12 years ago

    Benni,

    No, a FLASH_* object lives only in non-writeable address space. But if you have an array with only four int values, why don’t you just use a normal read/write array like this:

    int input_values[4];

    ?


  35. tinkerer

    12 years ago

    Hey Mikal,

    thanks a lot for your library, I’m using it in almost all my projects.
    Any chance of uploading your code to github? It would make keeping up to date with the latest version a lot easier.


  36. Chris

    12 years ago

    Just a quick suggestion to add the standard include guards in the library header, otherwise the compiler understandably complains about redefining previously defined items. Since the Arduino environment doesn’t have a good way of resolving library dependencies, inter-library includes must occur twice, once in the library and once in the main sketch.


  37. Anish

    12 years ago

    Hi Mikal ,
    I am not able to write in print.h in arduino 1.0 as it is in the core .
    So how do I put in the required patch ?

    Anish


  38. marcus

    12 years ago

    Hi Mikal,
    thanks for this library.

    If i compile with warnings enabled there is a warning for each created Flash object:

    warning: only initialized variables can be placed into program memory area

    turns out that this is a bug in the PROGMEM statement.

    I found that adding this in Flash.h takes away the warnings:

    // Workaround for http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34734
    #undef PROGMEM
    #define PROGMEM __attribute__((section(“.progmem.data”)))

    Maybe this is helpful for anyone…

    marcus


  39. marcus

    12 years ago

    Hi again,

    sorry but i just figured out that adding the two lines in my previous post result in a compile error:

    error: __c causes a section type conflict

    looks like it´s not that easy to fix…

    marcus


  40. Alexander Riccio

    12 years ago

    Fantastic library!!

    It’s working for what I’m doing, and my project is essentially finished. However, I am too OCD about compiler warnings to just accept that my code works.

    For every FLASH_STRING_ARRAY in my code, I get:

    //
    warning: ‘__progmem__’ attribute ignored

    warning: only initialized variables can be placed into program memory area
    […several more of these…]

    warning: comparison between signed and unsigned integer expressions
    //

    Also: I assume, that while crafting the miracle that is this library, you ignored these warnings for a reason, but I also get:

    //
    Flash.h: In member function ‘char* _FLASH_STRING::copy(char*, size_t, size_t) const’:
    Flash.h:75: warning: comparison between signed and unsigned integer expressions
    //


  41. Mikal

    12 years ago

    @Alexander,

    Thanks for the observations. I too am not too keen on the warnings that have starting cropping up with the newer compiler versions. I’ll see if I can’t cull those out in a future release.


  42. Tobias

    11 years ago

    Hi, very good library!!

    With Ubuntu 12.04 (avr-libc 1:1.7.1-2) compiling without errors. With Ubuntu 12.10 (avr-libc 1:1.8.0-2) i had many compiler errors. I changed all prog_char to char in Flash.cpp and Flash.h and compiling without errors.


  43. Kobus

    11 years ago

    Hi

    How do I pass the array to a function/method?

    I have this
    FLASH_ARRAY(int, BLUE_TEAM_SHOOT_BATTLE_1, 860, 260, 300, 260, 300, 540, 580, 260);

    I want to do this
    if (irCommons.compare(results, BLUE_TEAM_SHOOT_BATTLE_1)) {
    }

    bool IrCommons::compare(decode_results *results, int signal[])


  44. Mikal

    11 years ago

    @Kobus,

    The object defined by the FLASH_ARRAY macro is of type _FLASH_ARRAY, so you can declare your function:

    bool IrCommons:compare(decode_results *results, _FLASH_ARRAY &signal);


  45. The Q.

    11 years ago

    Mikal,
    Many thanks for this excellent library.
    It has saved me many hours and put my project back on track.
    Q..


  46. Mikal

    11 years ago

    @The Q,

    Great! Thanks!


  47. Devon

    11 years ago

    Hello Mikal,

    This is very interesting stuff, I’ve been trying to figure out the Flash/PROGMEM use for the last couple of weeks. I just found out that my compiler won’t verify/compile the examples you gave with the Flash4.zip.

    Any suggestions?

    Thanx


  48. Mikal

    11 years ago

    @Devon,

    What version/platform/board of Arduino are you using? What are the error messages?


  49. Fernando Garcia

    11 years ago

    Hello!

    I’m having problem of memory on my Arduino Mega 2560, I wonder if some function of this library can help me.

    At the moment I have an array with many strings that are saved in FLASH.

    The array is declared in the program this way:

    char* tabela_textos[] PROGMEM =
    {
    “”, // tabela_textos[0]
    “MENU 1”, // tabela_textos[1]
    “MENU 2”, // tabela_textos[2]
    “HORA E DATA”, // tabela_textos[3]
    ……..
    };

    I’ll need to print this string on my TFT.

    The library requires that use this kind of argument:

    ITDB02::print(char*, int, int, int)

    I am currently using this:

    print(tabela_textos[1], X1, Y1);

    You can find the full code here:

    https://github.com/FernandoGarcia/Ferduino_English

    The array with the string are here:

    https://github.com/FernandoGarcia/Ferduino_English/blob/master/Ferduino_English/A_Textos.ino

    Thank you for your attention.

    Sorry for english.

    Fernando Garcia

1 Trackbacks For This Post
  1. Arduino – Steuern per Internet | bastianmueller - blog

    […] Steckdose (An/Aus) hängt sich der Ardu komplett auf. Ihr benötigt zusätzlich die Bibliothek Flash (für das Speichern im Flash statt SRAM) und FreeMemory (für die Anzeige des freien […]

Leave a Reply