Use PSRAM

This article explains how to use the additional external RAM (PSRAM) available on some ESP32 boards (uPesy ESP32 Wrover DevKit, TinyPICO ) with the Arduino language and on the Framework ESP-IDF.

Overview

Since the internal RAM of the microcontrollers is quite low in general, an additional external RAM can be added. Even, if the ESP32 is a microcontroller that has a lot of RAM, it may not be sufficient, especially when you want to handle huge files JSON or HTML files

The PSRAM is an additional external RAM of 4 MB which is present in some ESP32 modules:
  • ESP32-WROVER-B

  • ESP32-WROVER-I

The ESP32 communicates with the PSRAM by SPI. This is why it is also called SPI RAM. The 3rd SPI bus of the ESP32 is used to communicate with the flash memory (which contains the program) and with the PSRAM.

../_images/psram.jpg

Internal overview of the ESP-WROVER-B module

Note

There is an ambiguity about the amount of real memory in the PSRAM. The real size of the PSRAM (on the silicon chip) is not 4MB but 8MB. However, only 4MB can be easily accessed by the software. To use the 8 MB of PSRAM, it is quite complex Use 8 MB of PSRAM . This said 4 MB more RAM is already huge (8 times the internal RAM)!

The advantage of an external RAM is to be able to relieve the internal RAM for the storage of important data:

We can for example:
  • Download large Json files

  • Store entire HTML WEB pages

  • Make a powerful web server

  • Create huge arrays

  • Read and manipulate data from large files from the SPIFFS or SD card

How to use it ?

On Arduino IDE, to be able to use the PSRAM, you have to select a compatible board, for example the ESP32 Wrover Module board which works for all ESP32 board that have PSRAM.

../_images/psram0.en.png

If you are using a board found in the list, such as for the TinyPICO board, check that the PSRAM is enabled.

../_images/psram1.en.png

Warning

The PSRAM does not appear in the ESP32 memory table, so it is normal that the size of the RAM indicated on Arduino IDE or Platform is always 327 KB. There is a clear separation between the internal RAM of the ESP32 (327 KB) and the external RAM (4 MB).

../_images/psram2.en.png

To use PSRAM, you can either use libraries that support PSRAM or use dynamic allocations to create arrays, buffers or download HTLM, JSON files …

Arduino / ESP32 libraries

To avoid to deal with dynamic allocations (and struggle), some Arduino libraries (compatible with ESP32) support PSRAM. The best known is ArduinoJson which allows you to easily handle JSON files on Arduino and ESP32.

../_images/arduinoJson.svg

ArduinoJson

If you are new to ArduinoJson, read this tutorial to install and learn how to use the library. To use the PSRAM with ArduinoJson, you must create a specific JsonDocument that is placed before the setup() function:

struct SpiRamAllocator {
        void* allocate(size_t size) {
                return ps_malloc(size);

        }
        void deallocate(void* pointer) {
                free(pointer);
        }
};

using SpiRamJsonDocument = BasicJsonDocument<SpiRamAllocator>;

Then we use the SpiRamJsonDocument like a classic DynamicJsonDocument:

SpiRamJsonDocument doc (100000); // 100 KB here
deserializeJson (doc, input);

Here is an example, which downloads a 65 KB JSON file. The JSON contains the results of the search “ESP32 json” on Google.

Example

#include <WiFi.h>
#include <HTTPClient.h>
#define ARDUINOJSON_DECODE_UNICODE 1
#include <ArduinoJson.h>

//WiFi
const char *ssid = "WiFi name";
const char *password = "WiFi password";

const char *url = "https://raw.githubusercontent.com/uPesy/ESP32_Tutorials/master/JSON/bigJsonExample.json";

struct SpiRamAllocator {
        void* allocate(size_t size) {
                return ps_malloc(size);

        }
        void deallocate(void* pointer) {
                free(pointer);
        }
};

using SpiRamJsonDocument = BasicJsonDocument<SpiRamAllocator>;


void setup(){
        delay(500);
        psramInit();
        Serial.begin(115200);
        Serial.println((String)"Memory available in PSRAM : " +ESP.getFreePsram());

        //Connect to WiFi

        WiFi.begin(ssid, password);
        while (WiFi.status() != WL_CONNECTED){
                delay(100);
                Serial.print(".");
        }

        Serial.print("\nWiFi connected with IP : ");
        Serial.println(WiFi.localIP());

        Serial.println("Downloading JSON");
        HTTPClient http;
        http.useHTTP10(true);
        http.begin(url);
        http.GET();

        SpiRamJsonDocument doc(100000); //Create a JSON document of 100 KB
        DeserializationError error = deserializeJson(doc, http.getStream());
        if(error){
                Serial.print("deserializeJson() failed: ");
                Serial.println(error.c_str());
        }

        http.end();
        Serial.println((String)"JsonDocument Usage Memory: " + doc.memoryUsage());


        for (int i=0; i!=10;i++){
                Serial.println("\n[+]");
                Serial.println(doc["items"][i]["title"].as<String>());
                Serial.println("------------------------------");
                Serial.println(doc["items"][i]["snippet"].as<String>());
                Serial.println("------------------------------");
                Serial.println((String) "URL : " + doc["items"][i]["link"].as<String>());

        }
}

void loop(){
}

Serial monitor:

Memory available in PSRAM : 4194252
..............
WiFi connected with IP : 192.168.43.7
Downloading JSON
JsonDocument Usage Memory: 62515

[+]
Installing ESP32 in Arduino IDE (Windows, Mac OS X, Linux ...
------------------------------
Installing ESP32 Add-on in Arduino IDE Windows, Mac OS X, Linux open. Enter
https://dl.espressif.com/dl/package_esp32_index.json into the “Additional Board ...
------------------------------
URL : https://randomnerdtutorials.com/installing-the-esp32-board-in-arduino-ide-windows-instructions/

[+]
ESP32: Parsing JSON – techtutorialsx
------------------------------
Apr 26, 2017 ... The objective of this post is to explain how to parse JSON messages with the
ESP32 and the ArduinoJson library. Introduction The objective of ...
------------------------------
URL : https://techtutorialsx.com/2017/04/26/esp32-parsing-json/

...

See my Instructable, that explains how you to use Google Search on an ESP32 and display the result in the serial monitor. (Also works without PSRAM)

../_images/psram3.jpg

See also

See how to use the external RAM on the ESP32 for JSON files on the official ArduinoJson site

Arduino framework

To store a variable in the PSRAM, dynamic allocation must be used with dedicated functions for the ESP32 PSRAM.

Note

It is not necessary to have advanced knowledge of C / C ++ to use PSRAM! Knowledge of pointers, dynamic allocation functions like malloc() or calloc() is a plus, but the examples below are sufficient in themselves to use PSRAM in electronic projects.

The main functions to use are the psramInit(), ESP.getFreePsram(), ps_malloc() or ps_calloc() and free().

The psramInit() function is used to initialize the PSRAM, the ESP.getFreePsram() function returns the amount of memory available in the PSRAM. The other 3 functions are used for dynamic allocation.

Examples

This section provides examples to use the PSRAM. They allow you to understand how to use the functions mentioned above. It is not necessary to understand everything to be able to use the PSRAM, it is enough to modify the values ​​in the examples to use them in your projects.

../_images/variable.svg
../_images/matrix.svg
../_images/string-text.svg
../_images/html.svg

Variable

Array

String

HTML / JSON

Variable
  • This example shows how to create variables that are stored in the the PSRAM:

//Create an integer nmumber
int *var_int = (int *) ps_malloc(sizeof(int));
*var_int = 42;

//Create a float number
float *var_float = (float *) ps_malloc(sizeof(float));
*var_float = 42.42;

Here is the full sketch:

Example

void setup() {
Serial.begin(115200);
//PSRAM Initialisation
if(psramInit()){
        Serial.println("\nThe PSRAM is correctly initialized");
}else{
        Serial.println("\nPSRAM does not work");
}

//Create an integer
int *var_int = (int *) ps_malloc(sizeof(int));
*var_int = 42;

//Create a float
float *var_float = (float *) ps_malloc(sizeof(float));
*var_float = 42.42;

Serial.println((String)"var_int = " + *var_int);
Serial.print("var_float = ");
Serial.println(*var_float);
}

Terminal output:

The PSRAM is correctly initialized
var_int = 42
var_float = 42.42
Array

For arrays, the syntax is very similar:

//Create an array of 1000 integers
int n_elements = 1000;
int *int_array = (int *) ps_malloc(n_elements * sizeof(int));
//We access array values like a classic array
int_array[0] = 42;
int_array[42] = 42;
int_array[999] = 42;
  • This the full sketch example that create an integer array of 1000 elements stored in the PSRAM:

Example

int n_elements = 1000;

void setup(){
        Serial.begin(115200);
        //Init
        if(psramInit()){
        Serial.println("\nPSRAM is correctly initialized");
        }else{
        Serial.println("PSRAM not available");
        }

        //Create an array of n_elements
        int available_PSRAM_size = ESP.getFreePsram();
        Serial.println((String)"PSRAM Size available (bytes): " + available_PSRAM_size);

        int *array_int = (int *) ps_malloc(n_elements * sizeof(int)); //Create an integer array of n_elements
        array_int[0] = 42;
        array_int[999] = 42; //We access array values like classic array

        int available_PSRAM_size_after = ESP.getFreePsram();
        Serial.println((String)"PSRAM Size available (bytes): " + available_PSRAM_size_after); // Free memory space has decreased
        int array_size = available_PSRAM_size - available_PSRAM_size_after;
        Serial.println((String)"Array size in PSRAM in bytes: " + array_size);

        //Delete array
        free(array_int); //The allocated memory is freed.
        Serial.println((String)"PSRAM Size available (bytes): " +ESP.getFreePsram());
}

void loop() {
}

Terminal output:

PSRAM is correctly initialized
PSRAM Size available (bytes): 4194252
PSRAM Size available (bytes): 4190236
Array size in PSRAM in bytes: 4016
PSRAM Size available (bytes): 4194252

Note

The size in bytes is 4016 for an array of 1000 integers (int). The type int is stored on 4 bytes. There are therefore 4 * 1000 bytes use by the array, the remaining 16 bytes contain information on the memory block (size, flags).

  • For other array type :

char * array1 = (char *) ps_malloc (n_elements * sizeof (char)); // Create an empty array of n_elements characters
float * array = (float *) ps_malloc (n_elements * sizeof (float)); // Create an array of n_elements float number
  • We can also create arrays that are filled with zeros with ps_calloc():

int n_elements = 20;
Serial.println((String)"PSRAM Size available (bytes): " +ESP.getFreePsram());
Serial.println("Array of integers initialized to 0");
int *array = (int *) ps_calloc(n_elements, sizeof(int));
Serial.print("[array] : ");
for(int i=0; i!= n_elements;i++){
        Serial.print((String)array[0] + " ");
}
Serial.println((String)"\nPSRAM Size available (bytes): " +ESP.getFreePsram());

Terminal output:

PSRAM Size available (bytes): 4194252
Array of integers initialized to 0
[array]: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
PSRAM Size available (bytes): 4194156
  • Here are 2 ways to create 2-dimensional arrays:

Example

int n_rows = 10;
int n_columns = 20;


void setup(){
        Serial.begin(115200);
        //Initialisation
        if(psramInit()){
                Serial.println("\nPSRAM is correctly initialized");
        }else{
                Serial.println("PSRAM not available");
        }

        //Create an array of n_elements
        int initialPSRAMSize = ESP.getFreePsram();
        Serial.println((String)"PSRAM Size available (bytes): " + initialPSRAMSize);
        //Creating a two-dimensional array with a one-dimensional array
        int *array2D = (int *) ps_malloc(n_rows * n_columns * sizeof(int)); // Create an array of n_rows x n_columns
        //To access the value located in row 5 to column 10
        int row_i = 5;
        int column_j = 10;
        array2D[column_j * n_columns + row_i] = 42;

        //Creating a two-dimensional zero-filled array with an array of pointers
        int **array2Dbis = (int **) ps_calloc(n_rows, sizeof(int *));
        for (int i =0; i!= n_columns;i++){
                array2Dbis[i] = (int *) ps_calloc(n_columns , sizeof(int));
        }
        //To access the value located in row 5 to column 10
        array2Dbis[row_i][column_j] = 42;


        int PSRAMSize = ESP.getFreePsram();
        Serial.println((String)"PSRAM Size available (bytes): " + PSRAMSize);
        int array2D_size = initialPSRAMSize - PSRAMSize;
        Serial.println((String)"Size of the array in PSRAM in bytes: " + array2D_size);

        //Delete 2D arrays
        free(array2D); //Allocated memory is freed.
        free(array2Dbis);
        Serial.println((String)"PSRAM Size available (bytes): " +ESP.getFreePsram());
}

void loop() {
}
String

String object is just a char array:

int n_elements = 20;
char *str = (char *) ps_calloc(n_elements, sizeof(char)); //Create an array of n_elements null characters ('\0')
str[0] = '4';
str[1] = '2';

Note

The use of ps_calloc() is more relevant here because it allows the ESP32, to directly detect the end of the string.

Here is an example sketch:

void setup() {
        Serial.begin(115200);
        if(psramInit()){
                Serial.println("\nThe PSRAM is correctly initialized");
        }else{
                Serial.println("\nPSRAM does not work");
        }

        int n_elements = 20;
        char *str = (char *) ps_calloc(n_elements, sizeof(char)); //Create an array of n_elements null characters ('\0')
        for(int i = 0; i < 10;i+=3){
                str[i] = '4';
                str[i+1] = '2';
                str[i+2] = '_';
        }
        Serial.println(str);
}

Terminal output:

The PSRAM is correctly initialized
42_42_42_42_
HTML / JSON

Here is an example which allows you to download any type of text files like HTLM pages or JSON files (less than 4MB) and store it in the PSRAM.

#include <WiFi.h>
#include <HTTPClient.h>
#define TIMEOUT 5000

const char *ssid = "SSID";
const char *password = "WiFi PASSWORD";
const char *url = "https://en.wikipedia.org/wiki/ESP32";

void setup() {
        Serial.begin(115200);

        //Connect to WiFi
        WiFi.begin(ssid, password);
        while (WiFi.status() != WL_CONNECTED){
        delay(100);
        Serial.print(".");
        }
        Serial.print("\nWiFi connected with IP : ");
        Serial.println(WiFi.localIP());

        char *page = (char *) ps_malloc(sizeof(char));
        int lengthHtml = 0;
        HTTPClient http;
        http.begin(url);
        http.useHTTP10(true); //Try to have a Content-Length in the header using HTTP1

        Serial.println((String) "Try GET "+ url);
        int httpCode = http.GET();
        if (httpCode > 0){
                Serial.printf("[HTTP] GET... code: %d\n", httpCode);
                if (httpCode == HTTP_CODE_OK){ // HTTP_CODE_OK = 200

                //Try to retrieve the size of the file (-1 if there is no Content-Length in the header)
                int tempLength = http.getSize();
                Serial.println((String) "Content Length :" + tempLength);

                Serial.printf("Memory address of HTML page in PSRAM : %p \n", page);

                WiFiClient *stream = http.getStreamPtr();

                int position = 0;

                uint32_t currentTime = 0;
                uint8_t timeoutArboted = 1;

                while(http.connected() && (tempLength > 0 || tempLength == -1)){

                        size_t size = stream->available();
                        if (size){
                                page = (char*) ps_realloc(page, position + size);
                                stream->readBytes(page+position, size);
                                position += size;

                                if (tempLength > 0){
                                        tempLength -= size;
                                }

                                timeoutArboted = 1;
                                }else{
                                        // If we do not know the size of the file, we assume that all the data is received before the timeout
                                        if(timeoutArboted){
                                                currentTime = millis();
                                                timeoutArboted = 0;
                                        }else{
                                                if(millis()-currentTime > TIMEOUT){
                                                        Serial.println("Timeout reached");
                                                        break;
                                                }
                                        }
                                }
                        }

                        *(page+position) = '\0';

                        lengthHtml = position;
                        Serial.println((String)"Downloaded " + lengthHtml + " Octets");
                }
        }
        else{
                Serial.printf("[HTTP] GET... failed, error: %s\n", http.errorToString(httpCode).c_str());
        }
        //Close HTTP Client
        http.end();
        delay(1500);
        Serial.println(page); // To display the entire HTML page

}

void loop(){}

Terminal output:

...........
WiFi connected with IP : 192.168.43.7
Try GET https://en.wikipedia.org/wiki/ESP32
[HTTP] GET... code: 200
Content Length :117615
Memory address of HTML page in PSRAM : 0x3f800024
Downloaded 117615 Octets
<!DOCTYPE html>
<html class="client-nojs" lang="en" dir="ltr">
<head>
<meta charset="UTF-8"/>
<title>ESP32 - Wikipedia</title>
...
...

Note

You can notice that it is much easier to use the ArduinoJson library to download JSON files.

Functions details

psramInit()
Returns

true if the PSRAM has successfully initialized, false otherwise

Return type

boolean

This function ensures that the PSRAM can be used by making various tests, then it initializes.

ESP.getFreePsram()
Returns

The memory available in bytes of the PSRAM.

Return type

int

This function allows you to find out the memory available in the PSRAM. The maximum size is 4,194,252 bytes.

ps_malloc(size_t tailleBloc)
Parameters
  • tailleBloc – Total number of bytes that you want to store in RAM.

Returns

A pointer of the type given in argument which contains the address of the memory block.

Return type

void *

This function allows you to allocate a memory block in the PSRAM.

ps_calloc(size_t nElements, int tailleElement)
Parameters
  • nElements – Number of elements that we want to store in the RAM

  • tailleElement – Size in bytes of a single element.

Returns

A pointer of the type given in argument which contains the address of the memory block.

Return type

void *

This function allocates a memory block of nElements, each of size sizeElement in the PSRAM and initializes all these bytes to the value of 0.

ps_realloc(void * adresseBloc, nouvelleTailleBloc)
Parameters
  • adresseBloc – Memory address of the memory block to be reallocated.

  • nouvelleTailleBloc – New size in bytes of the memory block that we want to reserve in RAM.

Returns

A pointer of the type given as an argument to the reallocated memory area.

Return type

void *

This function allows you to reallocate a memory block previously allocated with ps_malloc() or ps_calloc(). If the free memory space following the block to be reallocated is large enough for the new block size, the original memory block is simply enlarged. On the other hand, if the free space is not sufficient, a new memory block will be allocated, the content of the original zone copied into the new zone and the original memory block will be freed automatically.

free(void *pointer)
Parameters
  • pointer (*int, *char, *double, *long...) – The address of the memory block to be deallocated

This function frees a dynamically allocated memory block with ps_malloc() or ps_calloc().

ESP-IDF Framework

The functions used on the Arduino framework are based on those of the ESP-IDF Framework. Thus the functions described below are used implicitly on the Arduino-ESP32 Framework.

The functions to be used are the following esp_spiram_init(), heap_caps_get_free_size(), heap_caps_malloc() or heap_caps_calloc() and free()

See also

For further information, see the official documentation on PSRAM