How to pass large arrays of data from MATLAB to Python

Question

My data is stored in classes in MATLAB which contain large arrays but want to work with that data in python for training machine learning. From what I can tell, when moving data between the two, the value is transferred and so the array is copied in memory. Then converting the arrays into numpy arrays creates a further copy in memory. How can I avoid creating so copies of data as much as possible.

I have not yet implemented the python code as it will be pointless if I can't transfer it easily.

Answer 1

There are 3rd-party solutions which share memory between MATLAB/Python or native approaches as documented:

https://uk.mathworks.com/help/matlab/matlab_external/passing-data-to-python.html

Pass Matrices and Multidimensional Arrays to Python

The Python language provides a protocol for accessing memory buffers like the data stored in a MATLAB array. MATLAB implements this Python buffer protocol for MATLAB arrays so that you can read MATLAB arrays directly from Python code, running in the same process as MATLAB, without copying data.