forcing 32 bit access using python mmap?Reference a byte array as an integerCalling an external command in PythonWhat are metaclasses in Python?Finding the index of an item given a list containing it in PythonDifference between append vs. extend list methods in PythonHow can I safely create a nested directory in Python?Does Python have a ternary conditional operator?How to get the current time in PythonHow can I make a time delay in Python?Accessing the index in 'for' loops?Does Python have a string 'contains' substring method?
What is the difference between "Do you interest" and "...interested in" something?
Two-sided logarithm inequality
How do I repair my stair bannister?
Why is Arduino resetting while driving motors?
Some numbers are more equivalent than others
Have I saved too much for retirement so far?
Could the E-bike drivetrain wear down till needing replacement after 400 km?
Drawing a topological "handle" with Tikz
If a character with the Alert feat rolls a crit fail on their Perception check, are they surprised?
Why do IPv6 unique local addresses have to have a /48 prefix?
How should I respond when I lied about my education and the company finds out through background check?
Can a significant change in incentives void an employment contract?
Why has "pence" been used in this sentence, not "pences"?
Can someone explain how this makes sense electrically?
Visiting the UK as unmarried couple
Varistor? Purpose and principle
Can a Necromancer reuse the corpses left behind from slain undead?
How can Trident be so inexpensive? Will it orbit Triton or just do a (slow) flyby?
Do Legal Documents Require Signing In Standard Pen Colors?
How can "mimic phobia" be cured or prevented?
THT: What is a squared annular “ring”?
Bob has never been a M before
Can we have a perfect cadence in a minor key?
On a tidally locked planet, would time be quantized?
forcing 32 bit access using python mmap?
Reference a byte array as an integerCalling an external command in PythonWhat are metaclasses in Python?Finding the index of an item given a list containing it in PythonDifference between append vs. extend list methods in PythonHow can I safely create a nested directory in Python?Does Python have a ternary conditional operator?How to get the current time in PythonHow can I make a time delay in Python?Accessing the index in 'for' loops?Does Python have a string 'contains' substring method?
I run 64 bit python on a 64 bits arm processor.
One on the AXI bus of this processor is connected to a FPGA (which does bus and clock domain changes down to a 32 bit wide bus).
This piece of hardware does not like 64 bit accesses...
I am trying to access this FPGA via python mmap like this (within a class):
def __init__(self, base, len):
self.base = base
self.fd = open("/dev/mem", "r+")
self.lw = mmap.mmap(self.fd.fileno(),
len,
mmap.MAP_SHARED,
mmap.PROT_READ | mmap.PROT_WRITE,
offset=base)
def get_U32(self, offset):
s = self.lw[offset:offset+4]
return struct.unpack("<I", s)[0]
The idea was to that get_U32() would read a 32 bit word from the bus (hence the offset to offset+4 reading).
Sadly, it seems that mmap performs a 64 bit access to the bus anyway (some kind of caching for performance optimization I assume) and then performs the 32 bit "casting". The underlying FPGA is not happy...
In a C program, I simply write:
data = *((uint32_t *) address);
...and the CPU seems to gently perform a 32 bit access on its AXI bus, which the underlying hardware prefers....
(so right now, I have a (slow) workaround, where python requires a C program to interface the Hardware, via pipes)
Is there a way to force 64 bits python to perform a 32 bit access, as the previous C line obviously succeeds with?
The question is written about reading 32 bits here, but of course, writing 32 bits is needed as well...
python 32bit-64bit mmap
|
show 6 more comments
I run 64 bit python on a 64 bits arm processor.
One on the AXI bus of this processor is connected to a FPGA (which does bus and clock domain changes down to a 32 bit wide bus).
This piece of hardware does not like 64 bit accesses...
I am trying to access this FPGA via python mmap like this (within a class):
def __init__(self, base, len):
self.base = base
self.fd = open("/dev/mem", "r+")
self.lw = mmap.mmap(self.fd.fileno(),
len,
mmap.MAP_SHARED,
mmap.PROT_READ | mmap.PROT_WRITE,
offset=base)
def get_U32(self, offset):
s = self.lw[offset:offset+4]
return struct.unpack("<I", s)[0]
The idea was to that get_U32() would read a 32 bit word from the bus (hence the offset to offset+4 reading).
Sadly, it seems that mmap performs a 64 bit access to the bus anyway (some kind of caching for performance optimization I assume) and then performs the 32 bit "casting". The underlying FPGA is not happy...
In a C program, I simply write:
data = *((uint32_t *) address);
...and the CPU seems to gently perform a 32 bit access on its AXI bus, which the underlying hardware prefers....
(so right now, I have a (slow) workaround, where python requires a C program to interface the Hardware, via pipes)
Is there a way to force 64 bits python to perform a 32 bit access, as the previous C line obviously succeeds with?
The question is written about reading 32 bits here, but of course, writing 32 bits is needed as well...
python 32bit-64bit mmap
Are you sure about that? On the FPGA, looking at the final 32 bit bus, I see only one strobe using the C program, while I see 2 strobes when using python: My assumption is therefore that the C program only accesses 32 bits. As far as I understand, some signaling on AXI allows for shorter accesses than the max 64 bits. Maybe I am wrong, as I cannot see that bus, but C and python accesses are clearly different at the far end.
– user1159290
Mar 7 at 8:41
If that C program is 64-bit, it is still using 64-bit to read the address. The assumption that it has something to do with the bus size doesn't really make sense. Besides, everything you do in the application is in it's virtual memory space, you are not even dealing with physical memory addresses.
– Havenard
Mar 7 at 8:44
mmapuses the OS to access the memory. The C code is directly accessing a 32-bit quantity via its address. You're comparing apples and oranges.
– martineau
Mar 7 at 8:45
@martineau. Agreed. can you do "apple" accesses from python, then?
– user1159290
Mar 7 at 8:47
1
You might be able to do something like that with thedi()function shown in this answer. You might also want to try using a 32-bit version of the Python interpreter to run your script.
– martineau
Mar 7 at 8:51
|
show 6 more comments
I run 64 bit python on a 64 bits arm processor.
One on the AXI bus of this processor is connected to a FPGA (which does bus and clock domain changes down to a 32 bit wide bus).
This piece of hardware does not like 64 bit accesses...
I am trying to access this FPGA via python mmap like this (within a class):
def __init__(self, base, len):
self.base = base
self.fd = open("/dev/mem", "r+")
self.lw = mmap.mmap(self.fd.fileno(),
len,
mmap.MAP_SHARED,
mmap.PROT_READ | mmap.PROT_WRITE,
offset=base)
def get_U32(self, offset):
s = self.lw[offset:offset+4]
return struct.unpack("<I", s)[0]
The idea was to that get_U32() would read a 32 bit word from the bus (hence the offset to offset+4 reading).
Sadly, it seems that mmap performs a 64 bit access to the bus anyway (some kind of caching for performance optimization I assume) and then performs the 32 bit "casting". The underlying FPGA is not happy...
In a C program, I simply write:
data = *((uint32_t *) address);
...and the CPU seems to gently perform a 32 bit access on its AXI bus, which the underlying hardware prefers....
(so right now, I have a (slow) workaround, where python requires a C program to interface the Hardware, via pipes)
Is there a way to force 64 bits python to perform a 32 bit access, as the previous C line obviously succeeds with?
The question is written about reading 32 bits here, but of course, writing 32 bits is needed as well...
python 32bit-64bit mmap
I run 64 bit python on a 64 bits arm processor.
One on the AXI bus of this processor is connected to a FPGA (which does bus and clock domain changes down to a 32 bit wide bus).
This piece of hardware does not like 64 bit accesses...
I am trying to access this FPGA via python mmap like this (within a class):
def __init__(self, base, len):
self.base = base
self.fd = open("/dev/mem", "r+")
self.lw = mmap.mmap(self.fd.fileno(),
len,
mmap.MAP_SHARED,
mmap.PROT_READ | mmap.PROT_WRITE,
offset=base)
def get_U32(self, offset):
s = self.lw[offset:offset+4]
return struct.unpack("<I", s)[0]
The idea was to that get_U32() would read a 32 bit word from the bus (hence the offset to offset+4 reading).
Sadly, it seems that mmap performs a 64 bit access to the bus anyway (some kind of caching for performance optimization I assume) and then performs the 32 bit "casting". The underlying FPGA is not happy...
In a C program, I simply write:
data = *((uint32_t *) address);
...and the CPU seems to gently perform a 32 bit access on its AXI bus, which the underlying hardware prefers....
(so right now, I have a (slow) workaround, where python requires a C program to interface the Hardware, via pipes)
Is there a way to force 64 bits python to perform a 32 bit access, as the previous C line obviously succeeds with?
The question is written about reading 32 bits here, but of course, writing 32 bits is needed as well...
python 32bit-64bit mmap
python 32bit-64bit mmap
asked Mar 7 at 8:31
user1159290user1159290
367318
367318
Are you sure about that? On the FPGA, looking at the final 32 bit bus, I see only one strobe using the C program, while I see 2 strobes when using python: My assumption is therefore that the C program only accesses 32 bits. As far as I understand, some signaling on AXI allows for shorter accesses than the max 64 bits. Maybe I am wrong, as I cannot see that bus, but C and python accesses are clearly different at the far end.
– user1159290
Mar 7 at 8:41
If that C program is 64-bit, it is still using 64-bit to read the address. The assumption that it has something to do with the bus size doesn't really make sense. Besides, everything you do in the application is in it's virtual memory space, you are not even dealing with physical memory addresses.
– Havenard
Mar 7 at 8:44
mmapuses the OS to access the memory. The C code is directly accessing a 32-bit quantity via its address. You're comparing apples and oranges.
– martineau
Mar 7 at 8:45
@martineau. Agreed. can you do "apple" accesses from python, then?
– user1159290
Mar 7 at 8:47
1
You might be able to do something like that with thedi()function shown in this answer. You might also want to try using a 32-bit version of the Python interpreter to run your script.
– martineau
Mar 7 at 8:51
|
show 6 more comments
Are you sure about that? On the FPGA, looking at the final 32 bit bus, I see only one strobe using the C program, while I see 2 strobes when using python: My assumption is therefore that the C program only accesses 32 bits. As far as I understand, some signaling on AXI allows for shorter accesses than the max 64 bits. Maybe I am wrong, as I cannot see that bus, but C and python accesses are clearly different at the far end.
– user1159290
Mar 7 at 8:41
If that C program is 64-bit, it is still using 64-bit to read the address. The assumption that it has something to do with the bus size doesn't really make sense. Besides, everything you do in the application is in it's virtual memory space, you are not even dealing with physical memory addresses.
– Havenard
Mar 7 at 8:44
mmapuses the OS to access the memory. The C code is directly accessing a 32-bit quantity via its address. You're comparing apples and oranges.
– martineau
Mar 7 at 8:45
@martineau. Agreed. can you do "apple" accesses from python, then?
– user1159290
Mar 7 at 8:47
1
You might be able to do something like that with thedi()function shown in this answer. You might also want to try using a 32-bit version of the Python interpreter to run your script.
– martineau
Mar 7 at 8:51
Are you sure about that? On the FPGA, looking at the final 32 bit bus, I see only one strobe using the C program, while I see 2 strobes when using python: My assumption is therefore that the C program only accesses 32 bits. As far as I understand, some signaling on AXI allows for shorter accesses than the max 64 bits. Maybe I am wrong, as I cannot see that bus, but C and python accesses are clearly different at the far end.
– user1159290
Mar 7 at 8:41
Are you sure about that? On the FPGA, looking at the final 32 bit bus, I see only one strobe using the C program, while I see 2 strobes when using python: My assumption is therefore that the C program only accesses 32 bits. As far as I understand, some signaling on AXI allows for shorter accesses than the max 64 bits. Maybe I am wrong, as I cannot see that bus, but C and python accesses are clearly different at the far end.
– user1159290
Mar 7 at 8:41
If that C program is 64-bit, it is still using 64-bit to read the address. The assumption that it has something to do with the bus size doesn't really make sense. Besides, everything you do in the application is in it's virtual memory space, you are not even dealing with physical memory addresses.
– Havenard
Mar 7 at 8:44
If that C program is 64-bit, it is still using 64-bit to read the address. The assumption that it has something to do with the bus size doesn't really make sense. Besides, everything you do in the application is in it's virtual memory space, you are not even dealing with physical memory addresses.
– Havenard
Mar 7 at 8:44
mmap uses the OS to access the memory. The C code is directly accessing a 32-bit quantity via its address. You're comparing apples and oranges.– martineau
Mar 7 at 8:45
mmap uses the OS to access the memory. The C code is directly accessing a 32-bit quantity via its address. You're comparing apples and oranges.– martineau
Mar 7 at 8:45
@martineau. Agreed. can you do "apple" accesses from python, then?
– user1159290
Mar 7 at 8:47
@martineau. Agreed. can you do "apple" accesses from python, then?
– user1159290
Mar 7 at 8:47
1
1
You might be able to do something like that with the
di() function shown in this answer. You might also want to try using a 32-bit version of the Python interpreter to run your script.– martineau
Mar 7 at 8:51
You might be able to do something like that with the
di() function shown in this answer. You might also want to try using a 32-bit version of the Python interpreter to run your script.– martineau
Mar 7 at 8:51
|
show 6 more comments
1 Answer
1
active
oldest
votes
Based on the idea from @martineau, the double probe can be fixed using python ctypes, like:
s = ctypes.c_uint32.from_buffer(self.lw, offset).value #read
or
types.c_uint32.from_buffer(self.lw, offset).value = s #write
This does indeed seem to force python into doing the same 32 bit access as in C, and remove the double read or write probe at the 32 bit bus.
However, sadly, python seems to do a read before each write.
So the solution above works perfectly for reading, but when writing, I still get a read access before the write access. In C, I can, of course just get a single write access when writing.
I am posting this for others who may be interested. If you have a solution to this last issue (read before write), please post it.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55039249%2fforcing-32-bit-access-using-python-mmap%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Based on the idea from @martineau, the double probe can be fixed using python ctypes, like:
s = ctypes.c_uint32.from_buffer(self.lw, offset).value #read
or
types.c_uint32.from_buffer(self.lw, offset).value = s #write
This does indeed seem to force python into doing the same 32 bit access as in C, and remove the double read or write probe at the 32 bit bus.
However, sadly, python seems to do a read before each write.
So the solution above works perfectly for reading, but when writing, I still get a read access before the write access. In C, I can, of course just get a single write access when writing.
I am posting this for others who may be interested. If you have a solution to this last issue (read before write), please post it.
add a comment |
Based on the idea from @martineau, the double probe can be fixed using python ctypes, like:
s = ctypes.c_uint32.from_buffer(self.lw, offset).value #read
or
types.c_uint32.from_buffer(self.lw, offset).value = s #write
This does indeed seem to force python into doing the same 32 bit access as in C, and remove the double read or write probe at the 32 bit bus.
However, sadly, python seems to do a read before each write.
So the solution above works perfectly for reading, but when writing, I still get a read access before the write access. In C, I can, of course just get a single write access when writing.
I am posting this for others who may be interested. If you have a solution to this last issue (read before write), please post it.
add a comment |
Based on the idea from @martineau, the double probe can be fixed using python ctypes, like:
s = ctypes.c_uint32.from_buffer(self.lw, offset).value #read
or
types.c_uint32.from_buffer(self.lw, offset).value = s #write
This does indeed seem to force python into doing the same 32 bit access as in C, and remove the double read or write probe at the 32 bit bus.
However, sadly, python seems to do a read before each write.
So the solution above works perfectly for reading, but when writing, I still get a read access before the write access. In C, I can, of course just get a single write access when writing.
I am posting this for others who may be interested. If you have a solution to this last issue (read before write), please post it.
Based on the idea from @martineau, the double probe can be fixed using python ctypes, like:
s = ctypes.c_uint32.from_buffer(self.lw, offset).value #read
or
types.c_uint32.from_buffer(self.lw, offset).value = s #write
This does indeed seem to force python into doing the same 32 bit access as in C, and remove the double read or write probe at the 32 bit bus.
However, sadly, python seems to do a read before each write.
So the solution above works perfectly for reading, but when writing, I still get a read access before the write access. In C, I can, of course just get a single write access when writing.
I am posting this for others who may be interested. If you have a solution to this last issue (read before write), please post it.
answered Mar 8 at 7:32
user1159290user1159290
367318
367318
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55039249%2fforcing-32-bit-access-using-python-mmap%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are you sure about that? On the FPGA, looking at the final 32 bit bus, I see only one strobe using the C program, while I see 2 strobes when using python: My assumption is therefore that the C program only accesses 32 bits. As far as I understand, some signaling on AXI allows for shorter accesses than the max 64 bits. Maybe I am wrong, as I cannot see that bus, but C and python accesses are clearly different at the far end.
– user1159290
Mar 7 at 8:41
If that C program is 64-bit, it is still using 64-bit to read the address. The assumption that it has something to do with the bus size doesn't really make sense. Besides, everything you do in the application is in it's virtual memory space, you are not even dealing with physical memory addresses.
– Havenard
Mar 7 at 8:44
mmapuses the OS to access the memory. The C code is directly accessing a 32-bit quantity via its address. You're comparing apples and oranges.– martineau
Mar 7 at 8:45
@martineau. Agreed. can you do "apple" accesses from python, then?
– user1159290
Mar 7 at 8:47
1
You might be able to do something like that with the
di()function shown in this answer. You might also want to try using a 32-bit version of the Python interpreter to run your script.– martineau
Mar 7 at 8:51