Use PSRAM¶
This article explains how to use the additional external RAM (PSRAM) available on some ESP32 boards (uPesy ESP32 Wrover DevKit, TinyPICO ) with the Arduino language and on the Framework ESP-IDF.
Overview¶
Since the internal RAM of the microcontrollers is quite low in general, an additional external RAM can be added. Even, if the ESP32 is a microcontroller that has a lot of RAM, it may not be sufficient, especially when you want to handle huge files JSON or HTML files
- The PSRAM is an additional external RAM of 4 MB which is present in some ESP32 modules:
ESP32-WROVER-B
ESP32-WROVER-I
The ESP32 communicates with the PSRAM by SPI. This is why it is also called SPI RAM. The 3rd SPI bus of the ESP32 is used to communicate with the flash memory (which contains the program) and with the PSRAM.
Internal overview of the ESP-WROVER-B module¶
Note
There is an ambiguity about the amount of real memory in the PSRAM. The real size of the PSRAM (on the silicon chip) is not 4MB but 8MB. However, only 4MB can be easily accessed by the software. To use the 8 MB of PSRAM, it is quite complex Use 8 MB of PSRAM . This said 4 MB more RAM is already huge (8 times the internal RAM)!
The advantage of an external RAM is to be able to relieve the internal RAM for the storage of important data:
- We can for example:
Download large Json files
Store entire HTML WEB pages
Make a powerful web server
Create huge arrays
Read and manipulate data from large files from the SPIFFS or SD card
How to use it ?¶
On Arduino IDE, to be able to use the PSRAM, you have to select a compatible board, for example the ESP32 Wrover Module board which works for all ESP32 board that have PSRAM.
If you are using a board found in the list, such as for the TinyPICO board, check that the PSRAM is enabled.
Warning
The PSRAM does not appear in the ESP32 memory table, so it is normal that the size of the RAM indicated on Arduino IDE or Platform is always 327 KB. There is a clear separation between the internal RAM of the ESP32 (327 KB) and the external RAM (4 MB).
To use PSRAM, you can either use libraries that support PSRAM or use dynamic allocations to create arrays, buffers or download HTLM, JSON files …
Arduino / ESP32 libraries¶
To avoid to deal with dynamic allocations (and struggle), some Arduino libraries (compatible with ESP32) support PSRAM. The best known is ArduinoJson which allows you to easily handle JSON files on Arduino and ESP32.
ArduinoJson¶
If you are new to ArduinoJson, read this tutorial to install and learn how to use the library. To use the PSRAM with ArduinoJson, you must create a specific JsonDocument that is placed before the setup() function:
struct SpiRamAllocator {
void* allocate(size_t size) {
return ps_malloc(size);
}
void deallocate(void* pointer) {
free(pointer);
}
};
using SpiRamJsonDocument = BasicJsonDocument<SpiRamAllocator>;
Then we use the SpiRamJsonDocument like a classic DynamicJsonDocument:
SpiRamJsonDocument doc (100000); // 100 KB here
deserializeJson (doc, input);
Here is an example, which downloads a 65 KB JSON file. The JSON contains the results of the search “ESP32 json” on Google.
Example
#include <WiFi.h> #include <HTTPClient.h> #define ARDUINOJSON_DECODE_UNICODE 1 #include <ArduinoJson.h> //WiFi const char *ssid = "WiFi name"; const char *password = "WiFi password"; const char *url = "https://raw.githubusercontent.com/uPesy/ESP32_Tutorials/master/JSON/bigJsonExample.json"; struct SpiRamAllocator { void* allocate(size_t size) { return ps_malloc(size); } void deallocate(void* pointer) { free(pointer); } }; using SpiRamJsonDocument = BasicJsonDocument<SpiRamAllocator>; void setup(){ delay(500); psramInit(); Serial.begin(115200); Serial.println((String)"Memory available in PSRAM : " +ESP.getFreePsram()); //Connect to WiFi WiFi.begin(ssid, password); while (WiFi.status() != WL_CONNECTED){ delay(100); Serial.print("."); } Serial.print("\nWiFi connected with IP : "); Serial.println(WiFi.localIP()); Serial.println("Downloading JSON"); HTTPClient http; http.useHTTP10(true); http.begin(url); http.GET(); SpiRamJsonDocument doc(100000); //Create a JSON document of 100 KB DeserializationError error = deserializeJson(doc, http.getStream()); if(error){ Serial.print("deserializeJson() failed: "); Serial.println(error.c_str()); } http.end(); Serial.println((String)"JsonDocument Usage Memory: " + doc.memoryUsage()); for (int i=0; i!=10;i++){ Serial.println("\n[+]"); Serial.println(doc["items"][i]["title"].as<String>()); Serial.println("------------------------------"); Serial.println(doc["items"][i]["snippet"].as<String>()); Serial.println("------------------------------"); Serial.println((String) "URL : " + doc["items"][i]["link"].as<String>()); } } void loop(){ }
Serial monitor:
Memory available in PSRAM : 4194252
..............
WiFi connected with IP : 192.168.43.7
Downloading JSON
JsonDocument Usage Memory: 62515
[+]
Installing ESP32 in Arduino IDE (Windows, Mac OS X, Linux ...
------------------------------
Installing ESP32 Add-on in Arduino IDE Windows, Mac OS X, Linux open. Enter
https://dl.espressif.com/dl/package_esp32_index.json into the “Additional Board ...
------------------------------
URL : https://randomnerdtutorials.com/installing-the-esp32-board-in-arduino-ide-windows-instructions/
[+]
ESP32: Parsing JSON – techtutorialsx
------------------------------
Apr 26, 2017 ... The objective of this post is to explain how to parse JSON messages with the
ESP32 and the ArduinoJson library. Introduction The objective of ...
------------------------------
URL : https://techtutorialsx.com/2017/04/26/esp32-parsing-json/
...
See my Instructable, that explains how you to use Google Search on an ESP32 and display the result in the serial monitor. (Also works without PSRAM)
See also
See how to use the external RAM on the ESP32 for JSON files on the official ArduinoJson site
Arduino framework¶
To store a variable in the PSRAM, dynamic allocation must be used with dedicated functions for the ESP32 PSRAM.
Note
It is not necessary to have advanced knowledge of C / C ++ to use PSRAM! Knowledge of pointers, dynamic allocation functions like malloc() or calloc() is a plus, but the examples below are sufficient in themselves to use PSRAM in electronic projects.
The main functions to use are the psramInit(), ESP.getFreePsram(), ps_malloc() or ps_calloc() and free().
The psramInit() function is used to initialize the PSRAM, the ESP.getFreePsram() function returns the amount of memory available in the PSRAM. The other 3 functions are used for dynamic allocation.
Examples¶
This section provides examples to use the PSRAM. They allow you to understand how to use the functions mentioned above. It is not necessary to understand everything to be able to use the PSRAM, it is enough to modify the values in the examples to use them in your projects.
|
|
|
|
|---|---|---|---|
Variable¶
This example shows how to create variables that are stored in the the PSRAM:
//Create an integer nmumber
int *var_int = (int *) ps_malloc(sizeof(int));
*var_int = 42;
//Create a float number
float *var_float = (float *) ps_malloc(sizeof(float));
*var_float = 42.42;
Here is the full sketch:
Example
void setup() { Serial.begin(115200); //PSRAM Initialisation if(psramInit()){ Serial.println("\nThe PSRAM is correctly initialized"); }else{ Serial.println("\nPSRAM does not work"); } //Create an integer int *var_int = (int *) ps_malloc(sizeof(int)); *var_int = 42; //Create a float float *var_float = (float *) ps_malloc(sizeof(float)); *var_float = 42.42; Serial.println((String)"var_int = " + *var_int); Serial.print("var_float = "); Serial.println(*var_float); }Terminal output:
The PSRAM is correctly initialized var_int = 42 var_float = 42.42
Array¶
For arrays, the syntax is very similar:
//Create an array of 1000 integers
int n_elements = 1000;
int *int_array = (int *) ps_malloc(n_elements * sizeof(int));
//We access array values like a classic array
int_array[0] = 42;
int_array[42] = 42;
int_array[999] = 42;
This the full sketch example that create an integer array of 1000 elements stored in the PSRAM:
Example
int n_elements = 1000; void setup(){ Serial.begin(115200); //Init if(psramInit()){ Serial.println("\nPSRAM is correctly initialized"); }else{ Serial.println("PSRAM not available"); } //Create an array of n_elements int available_PSRAM_size = ESP.getFreePsram(); Serial.println((String)"PSRAM Size available (bytes): " + available_PSRAM_size); int *array_int = (int *) ps_malloc(n_elements * sizeof(int)); //Create an integer array of n_elements array_int[0] = 42; array_int[999] = 42; //We access array values like classic array int available_PSRAM_size_after = ESP.getFreePsram(); Serial.println((String)"PSRAM Size available (bytes): " + available_PSRAM_size_after); // Free memory space has decreased int array_size = available_PSRAM_size - available_PSRAM_size_after; Serial.println((String)"Array size in PSRAM in bytes: " + array_size); //Delete array free(array_int); //The allocated memory is freed. Serial.println((String)"PSRAM Size available (bytes): " +ESP.getFreePsram()); } void loop() { }Terminal output:
PSRAM is correctly initialized PSRAM Size available (bytes): 4194252 PSRAM Size available (bytes): 4190236 Array size in PSRAM in bytes: 4016 PSRAM Size available (bytes): 4194252
Note
The size in bytes is 4016 for an array of 1000 integers (int). The type int is stored on 4 bytes. There are therefore 4 * 1000 bytes use by the array, the remaining 16 bytes contain information on the memory block (size, flags).
For other array type :
char * array1 = (char *) ps_malloc (n_elements * sizeof (char)); // Create an empty array of n_elements characters
float * array = (float *) ps_malloc (n_elements * sizeof (float)); // Create an array of n_elements float number
We can also create arrays that are filled with zeros with
ps_calloc():
int n_elements = 20;
Serial.println((String)"PSRAM Size available (bytes): " +ESP.getFreePsram());
Serial.println("Array of integers initialized to 0");
int *array = (int *) ps_calloc(n_elements, sizeof(int));
Serial.print("[array] : ");
for(int i=0; i!= n_elements;i++){
Serial.print((String)array[0] + " ");
}
Serial.println((String)"\nPSRAM Size available (bytes): " +ESP.getFreePsram());
Terminal output:
PSRAM Size available (bytes): 4194252
Array of integers initialized to 0
[array]: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
PSRAM Size available (bytes): 4194156
Here are 2 ways to create 2-dimensional arrays:
Example
int n_rows = 10; int n_columns = 20; void setup(){ Serial.begin(115200); //Initialisation if(psramInit()){ Serial.println("\nPSRAM is correctly initialized"); }else{ Serial.println("PSRAM not available"); } //Create an array of n_elements int initialPSRAMSize = ESP.getFreePsram(); Serial.println((String)"PSRAM Size available (bytes): " + initialPSRAMSize); //Creating a two-dimensional array with a one-dimensional array int *array2D = (int *) ps_malloc(n_rows * n_columns * sizeof(int)); // Create an array of n_rows x n_columns //To access the value located in row 5 to column 10 int row_i = 5; int column_j = 10; array2D[column_j * n_columns + row_i] = 42; //Creating a two-dimensional zero-filled array with an array of pointers int **array2Dbis = (int **) ps_calloc(n_rows, sizeof(int *)); for (int i =0; i!= n_columns;i++){ array2Dbis[i] = (int *) ps_calloc(n_columns , sizeof(int)); } //To access the value located in row 5 to column 10 array2Dbis[row_i][column_j] = 42; int PSRAMSize = ESP.getFreePsram(); Serial.println((String)"PSRAM Size available (bytes): " + PSRAMSize); int array2D_size = initialPSRAMSize - PSRAMSize; Serial.println((String)"Size of the array in PSRAM in bytes: " + array2D_size); //Delete 2D arrays free(array2D); //Allocated memory is freed. free(array2Dbis); Serial.println((String)"PSRAM Size available (bytes): " +ESP.getFreePsram()); } void loop() { }
String¶
String object is just a char array:
int n_elements = 20;
char *str = (char *) ps_calloc(n_elements, sizeof(char)); //Create an array of n_elements null characters ('\0')
str[0] = '4';
str[1] = '2';
Note
The use of ps_calloc() is more relevant here because it allows the ESP32, to directly detect the end of the string.
Here is an example sketch:
void setup() {
Serial.begin(115200);
if(psramInit()){
Serial.println("\nThe PSRAM is correctly initialized");
}else{
Serial.println("\nPSRAM does not work");
}
int n_elements = 20;
char *str = (char *) ps_calloc(n_elements, sizeof(char)); //Create an array of n_elements null characters ('\0')
for(int i = 0; i < 10;i+=3){
str[i] = '4';
str[i+1] = '2';
str[i+2] = '_';
}
Serial.println(str);
}
Terminal output:
The PSRAM is correctly initialized
42_42_42_42_
HTML / JSON¶
Voici un exemple qui permet de télécharger n’importe quel type de fichiers textes comme des pages HTML ou des fichiers JSON (de moins de 4 Mo) et de le stocker dans la PSRAM.
#include <WiFi.h>
#include <HTTPClient.h>
#define TIMEOUT 5000
const char *ssid = "Nom Box WiFi";
const char *password = "mdp";
const char *url = "https://fr.wikipedia.org/wiki/ESP32";
void setup() {
Serial.begin(115200);
//Connect to WiFi
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED){
delay(100);
Serial.print(".");
}
Serial.print("\nConnecté au WiFi avec comme IP : ");
Serial.println(WiFi.localIP());
char *page = (char *) ps_malloc(sizeof(char));
int lengthHtml = 0;
HTTPClient http;
http.begin(url);
http.useHTTP10(true); //Permet d'essayer d'avoir un Content-Length dans le header en utilisant HTTP1
Serial.println((String) "Try GET "+ url);
int httpCode = http.GET();
if (httpCode > 0){
// Affiche le réponse le code HTTP de la requete GET
Serial.printf("[HTTP] GET... code: %d\n", httpCode);
// Si le fichier existe (HTML, JSON, ...)
if (httpCode == HTTP_CODE_OK){ // HTTP_CODE_OK = 200
// Essaye de récupérer la taille du ficher ( -1 s'il n'y a pas de Content-Length dans l'header)
int tempLength = http.getSize();
Serial.println((String) "Content Length :" + tempLength);
//Stockage de la page dans la PSRAM
Serial.printf("Adresse mémoire de la page : %p \n", page);
// Récupère le stream TCP
WiFiClient *stream = http.getStreamPtr();
//Initialisation position du buffer dans la PSRAM
int position = 0;
uint32_t currentTime = 0;
uint8_t timeoutArboted = 1;
// Récupère toute les données du fichier
while(http.connected() && (tempLength > 0 || tempLength == -1)){
// Récupère les données disponibles (les données arrivent packets par packets)
size_t size = stream->available();
if (size){
page = (char*) ps_realloc(page, position + size + 1);
stream->readBytes(page+position, size);
position += size;
if (tempLength > 0){
tempLength -= size;
}
timeoutArboted = 1;
}else{
//Si on ne connaît pas la taille du fichier, on suppose que toutes les données sont recues avant le timeout
if(timeoutArboted){
//Lance le timer
currentTime = millis();
timeoutArboted = 0;
}else{
if(millis()-currentTime > TIMEOUT){
//On a atteind le Timeout
Serial.println("Timeout reached");
break;
}
}
}
}
*(page+position) = '\0';
lengthHtml = position;
Serial.println((String)"Downloaded " + lengthHtml + " Octets");
}
}
else{
Serial.printf("[HTTP] GET... failed, error: %s\n", http.errorToString(httpCode).c_str());
}
//Ferme le client HTTP
http.end();
delay(1500);
Serial.println(page); // Pour afficher la page HTML entière
}
Terminal output:
...........
WiFi connected with IP : 192.168.43.7
Try GET https://en.wikipedia.org/wiki/ESP32
[HTTP] GET... code: 200
Content Length :117615
Memory address of HTML page in PSRAM : 0x3f800024
Downloaded 117615 Octets
<!DOCTYPE html>
<html class="client-nojs" lang="en" dir="ltr">
<head>
<meta charset="UTF-8"/>
<title>ESP32 - Wikipedia</title>
...
...
Note
You can notice that it is much easier to use the ArduinoJson library to download JSON files.
Functions details¶
-
psramInit() - Returns
true if the PSRAM has successfully initialized, false otherwise
- Return type
boolean
This function ensures that the PSRAM can be used by making various tests, then it initializes.
-
ESP.getFreePsram() - Returns
The memory available in bytes of the PSRAM.
- Return type
int
This function allows you to find out the memory available in the PSRAM. The maximum size is 4,194,252 bytes.
-
ps_malloc(size_t tailleBloc) - Parameters
tailleBloc – Total number of bytes that you want to store in RAM.
- Returns
A pointer of the type given in argument which contains the address of the memory block.
- Return type
void *
This function allows you to allocate a memory block in the PSRAM.
-
ps_calloc(size_t nElements, int tailleElement) - Parameters
nElements – Number of elements that we want to store in the RAM
tailleElement – Size in bytes of a single element.
- Returns
A pointer of the type given in argument which contains the address of the memory block.
- Return type
void *
This function allocates a memory block of nElements, each of size sizeElement in the PSRAM and initializes all these bytes to the value of 0.
-
ps_realloc(void * adresseBloc, nouvelleTailleBloc) - Parameters
adresseBloc – Memory address of the memory block to be reallocated.
nouvelleTailleBloc – New size in bytes of the memory block that we want to reserve in RAM.
- Returns
A pointer of the type given as an argument to the reallocated memory area.
- Return type
void *
This function allows you to reallocate a memory block previously allocated with ps_malloc() or ps_calloc(). If the free memory space following the block to be reallocated is large enough for the new block size, the original memory block is simply enlarged. On the other hand, if the free space is not sufficient, a new memory block will be allocated, the content of the original zone copied into the new zone and the original memory block will be freed automatically.
-
free(void *pointer) - Parameters
pointer (*int, *char, *double, *long...) – The address of the memory block to be deallocated
This function frees a dynamically allocated memory block with ps_malloc() or ps_calloc().
ESP-IDF Framework¶
The functions used on the Arduino framework are based on those of the ESP-IDF Framework. Thus the functions described below are used implicitly on the Arduino-ESP32 Framework.
The functions to be used are the following esp_spiram_init(), heap_caps_get_free_size(), heap_caps_malloc() or heap_caps_calloc() and free()
See also
For further information, see the official documentation on PSRAM