Parsing datastore Entity in Python efficiently The 2019 Stack Overflow Developer Survey Results Are InAppengine datastore phantom entity - inconsistent state?Best practice to query large number of ndb entities from datastoreappengine: How to query entities by key, in entities with ancester?Deleted entities keep returning using datastore adminGAE datastore admin copy failing on MapReduce model to JSON conversionHow to avoid “safety” over quota panic when accessing datastore ? (billing is enabled)Querying immediately for saved entity returns null sometimes - datastoregoogle datastore: breaking change re: anonymous struct fields?How to filter a parent entity using properties of child entity in datastoreGoogle Cloud Datastore Indexes for count queries
Understanding the implication of what "well-defined" means for the operation in quotient group
Why isn't airport relocation done gradually?
Lethal sonic weapons
aging parents with no investments
How to deal with fear of taking dependencies
Why is my p-value correlated to difference between means in two sample tests?
Why is it "Tumoren" and not "Tumore"?
What are the motivations for publishing new editions of an existing textbook, beyond new discoveries in a field?
What is the meaning of Triage in Cybersec world?
Is this food a bread or a loaf?
Which Sci-Fi work first showed weapon of galactic-scale mass destruction?
What do the Banks children have against barley water?
Why do UK politicians seemingly ignore opinion polls on Brexit?
What can other administrators access on my machine?
What does Linus Torvalds mean when he says that Git "never ever" tracks a file?
Where does the "burst of radiance" from Holy Weapon originate?
How to make payment on the internet without leaving a money trail?
Does duplicating a spell with Wish count as casting that spell?
What is the best strategy for white in this position?
Is flight data recorder erased after every flight?
Does light intensity oscillate really fast since it is a wave?
"What time...?" or "At what time...?" - what is more grammatically correct?
What does "rabbited" mean/imply in this sentence?
Why don't Unix/Linux systems traverse through directories until they find the required version of a linked library?
Parsing datastore Entity in Python efficiently
The 2019 Stack Overflow Developer Survey Results Are InAppengine datastore phantom entity - inconsistent state?Best practice to query large number of ndb entities from datastoreappengine: How to query entities by key, in entities with ancester?Deleted entities keep returning using datastore adminGAE datastore admin copy failing on MapReduce model to JSON conversionHow to avoid “safety” over quota panic when accessing datastore ? (billing is enabled)Querying immediately for saved entity returns null sometimes - datastoregoogle datastore: breaking change re: anonymous struct fields?How to filter a parent entity using properties of child entity in datastoreGoogle Cloud Datastore Indexes for count queries
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
So right now, datastore ds.query(kind='users')
returns me a response in the form:
<Entity(u'users', 5633378543992832L) u'username': u'xyz', u'user_ip': '127.0.0.1', u'name': u'xyz', u'password': 'something', u'register_date': datetime.datetime(2019, 3, 8, 5, 50, 1, 443212, tzinfo=<UTC>)>
And although I can iterate it like this:
result =
for oneItem in query.fetch():
# oneItem is Entity iterable as shown above
for oneProp in oneItem:
result[oneProp] = oneItem[oneProp]
And access any property by something = result['password']
Which works fine but it is grossly inefficient. Is there any way I can directly access any specific property without using for loops or another data structure? Similar to accessing a value in JSON.
I am using from google.cloud import datastore
firebase nosql google-cloud-datastore datastore
add a comment |
So right now, datastore ds.query(kind='users')
returns me a response in the form:
<Entity(u'users', 5633378543992832L) u'username': u'xyz', u'user_ip': '127.0.0.1', u'name': u'xyz', u'password': 'something', u'register_date': datetime.datetime(2019, 3, 8, 5, 50, 1, 443212, tzinfo=<UTC>)>
And although I can iterate it like this:
result =
for oneItem in query.fetch():
# oneItem is Entity iterable as shown above
for oneProp in oneItem:
result[oneProp] = oneItem[oneProp]
And access any property by something = result['password']
Which works fine but it is grossly inefficient. Is there any way I can directly access any specific property without using for loops or another data structure? Similar to accessing a value in JSON.
I am using from google.cloud import datastore
firebase nosql google-cloud-datastore datastore
What are you trying to illustrate in your example? Your for-loop keeps overwriting the values inresult
with the next entity. You are effectively just doing this:result = query.fetch()[-1]
– Alex
Mar 8 at 16:52
ahh yes, in my case it would only ever have one entity which again I can't, or don't know, how to access directly. query.fetch() doesn't support indexing. I wouldn't be posting this question if it did.
– Zain Qasmi
Mar 9 at 0:16
add a comment |
So right now, datastore ds.query(kind='users')
returns me a response in the form:
<Entity(u'users', 5633378543992832L) u'username': u'xyz', u'user_ip': '127.0.0.1', u'name': u'xyz', u'password': 'something', u'register_date': datetime.datetime(2019, 3, 8, 5, 50, 1, 443212, tzinfo=<UTC>)>
And although I can iterate it like this:
result =
for oneItem in query.fetch():
# oneItem is Entity iterable as shown above
for oneProp in oneItem:
result[oneProp] = oneItem[oneProp]
And access any property by something = result['password']
Which works fine but it is grossly inefficient. Is there any way I can directly access any specific property without using for loops or another data structure? Similar to accessing a value in JSON.
I am using from google.cloud import datastore
firebase nosql google-cloud-datastore datastore
So right now, datastore ds.query(kind='users')
returns me a response in the form:
<Entity(u'users', 5633378543992832L) u'username': u'xyz', u'user_ip': '127.0.0.1', u'name': u'xyz', u'password': 'something', u'register_date': datetime.datetime(2019, 3, 8, 5, 50, 1, 443212, tzinfo=<UTC>)>
And although I can iterate it like this:
result =
for oneItem in query.fetch():
# oneItem is Entity iterable as shown above
for oneProp in oneItem:
result[oneProp] = oneItem[oneProp]
And access any property by something = result['password']
Which works fine but it is grossly inefficient. Is there any way I can directly access any specific property without using for loops or another data structure? Similar to accessing a value in JSON.
I am using from google.cloud import datastore
firebase nosql google-cloud-datastore datastore
firebase nosql google-cloud-datastore datastore
edited Mar 9 at 1:02
Zain Qasmi
asked Mar 8 at 8:26
Zain QasmiZain Qasmi
387
387
What are you trying to illustrate in your example? Your for-loop keeps overwriting the values inresult
with the next entity. You are effectively just doing this:result = query.fetch()[-1]
– Alex
Mar 8 at 16:52
ahh yes, in my case it would only ever have one entity which again I can't, or don't know, how to access directly. query.fetch() doesn't support indexing. I wouldn't be posting this question if it did.
– Zain Qasmi
Mar 9 at 0:16
add a comment |
What are you trying to illustrate in your example? Your for-loop keeps overwriting the values inresult
with the next entity. You are effectively just doing this:result = query.fetch()[-1]
– Alex
Mar 8 at 16:52
ahh yes, in my case it would only ever have one entity which again I can't, or don't know, how to access directly. query.fetch() doesn't support indexing. I wouldn't be posting this question if it did.
– Zain Qasmi
Mar 9 at 0:16
What are you trying to illustrate in your example? Your for-loop keeps overwriting the values in
result
with the next entity. You are effectively just doing this: result = query.fetch()[-1]
– Alex
Mar 8 at 16:52
What are you trying to illustrate in your example? Your for-loop keeps overwriting the values in
result
with the next entity. You are effectively just doing this: result = query.fetch()[-1]
– Alex
Mar 8 at 16:52
ahh yes, in my case it would only ever have one entity which again I can't, or don't know, how to access directly. query.fetch() doesn't support indexing. I wouldn't be posting this question if it did.
– Zain Qasmi
Mar 9 at 0:16
ahh yes, in my case it would only ever have one entity which again I can't, or don't know, how to access directly. query.fetch() doesn't support indexing. I wouldn't be posting this question if it did.
– Zain Qasmi
Mar 9 at 0:16
add a comment |
1 Answer
1
active
oldest
votes
Ideally you'd be retrieving the desired object by key, instead of loading up the whole table.
It's a little challenging to give you exact code since I can't tell which library you are using to access the datastore. I've always used ndb
but it seems like you are using this one:
https://googleapis.github.io/google-cloud-python/latest/datastore/index.html
To fetch by key you'd do:
from google.cloud import datastore
from google.cloud.datastore.key import Key
ds = datastore.Client()
oneItem = ds.get(Key(u'users', 5633378543992832L, project=project))
and then at this point, just interact with it oneItem['password']
The id 5633378543992832L
should be supplied to you from the current session. So you'd only need to query during session creation. Something like this:
def create_session(username, raw_password):
client = datastore.Client()
query = client.query(kind=u'users')
query.add_filter('username', '=', username)
query.add_filter('password', '=', _your_password_hash_function(raw_password))
results = query.fetch(1)
if results:
return _create_session_for_user(results[0])
raise Exception("Invalid username/password")
You'll need add an index for the above query to work. It seems like your not using app engine, so you probably have to add indices through the web console
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55059285%2fparsing-datastore-entity-in-python-efficiently%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Ideally you'd be retrieving the desired object by key, instead of loading up the whole table.
It's a little challenging to give you exact code since I can't tell which library you are using to access the datastore. I've always used ndb
but it seems like you are using this one:
https://googleapis.github.io/google-cloud-python/latest/datastore/index.html
To fetch by key you'd do:
from google.cloud import datastore
from google.cloud.datastore.key import Key
ds = datastore.Client()
oneItem = ds.get(Key(u'users', 5633378543992832L, project=project))
and then at this point, just interact with it oneItem['password']
The id 5633378543992832L
should be supplied to you from the current session. So you'd only need to query during session creation. Something like this:
def create_session(username, raw_password):
client = datastore.Client()
query = client.query(kind=u'users')
query.add_filter('username', '=', username)
query.add_filter('password', '=', _your_password_hash_function(raw_password))
results = query.fetch(1)
if results:
return _create_session_for_user(results[0])
raise Exception("Invalid username/password")
You'll need add an index for the above query to work. It seems like your not using app engine, so you probably have to add indices through the web console
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
add a comment |
Ideally you'd be retrieving the desired object by key, instead of loading up the whole table.
It's a little challenging to give you exact code since I can't tell which library you are using to access the datastore. I've always used ndb
but it seems like you are using this one:
https://googleapis.github.io/google-cloud-python/latest/datastore/index.html
To fetch by key you'd do:
from google.cloud import datastore
from google.cloud.datastore.key import Key
ds = datastore.Client()
oneItem = ds.get(Key(u'users', 5633378543992832L, project=project))
and then at this point, just interact with it oneItem['password']
The id 5633378543992832L
should be supplied to you from the current session. So you'd only need to query during session creation. Something like this:
def create_session(username, raw_password):
client = datastore.Client()
query = client.query(kind=u'users')
query.add_filter('username', '=', username)
query.add_filter('password', '=', _your_password_hash_function(raw_password))
results = query.fetch(1)
if results:
return _create_session_for_user(results[0])
raise Exception("Invalid username/password")
You'll need add an index for the above query to work. It seems like your not using app engine, so you probably have to add indices through the web console
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
add a comment |
Ideally you'd be retrieving the desired object by key, instead of loading up the whole table.
It's a little challenging to give you exact code since I can't tell which library you are using to access the datastore. I've always used ndb
but it seems like you are using this one:
https://googleapis.github.io/google-cloud-python/latest/datastore/index.html
To fetch by key you'd do:
from google.cloud import datastore
from google.cloud.datastore.key import Key
ds = datastore.Client()
oneItem = ds.get(Key(u'users', 5633378543992832L, project=project))
and then at this point, just interact with it oneItem['password']
The id 5633378543992832L
should be supplied to you from the current session. So you'd only need to query during session creation. Something like this:
def create_session(username, raw_password):
client = datastore.Client()
query = client.query(kind=u'users')
query.add_filter('username', '=', username)
query.add_filter('password', '=', _your_password_hash_function(raw_password))
results = query.fetch(1)
if results:
return _create_session_for_user(results[0])
raise Exception("Invalid username/password")
You'll need add an index for the above query to work. It seems like your not using app engine, so you probably have to add indices through the web console
Ideally you'd be retrieving the desired object by key, instead of loading up the whole table.
It's a little challenging to give you exact code since I can't tell which library you are using to access the datastore. I've always used ndb
but it seems like you are using this one:
https://googleapis.github.io/google-cloud-python/latest/datastore/index.html
To fetch by key you'd do:
from google.cloud import datastore
from google.cloud.datastore.key import Key
ds = datastore.Client()
oneItem = ds.get(Key(u'users', 5633378543992832L, project=project))
and then at this point, just interact with it oneItem['password']
The id 5633378543992832L
should be supplied to you from the current session. So you'd only need to query during session creation. Something like this:
def create_session(username, raw_password):
client = datastore.Client()
query = client.query(kind=u'users')
query.add_filter('username', '=', username)
query.add_filter('password', '=', _your_password_hash_function(raw_password))
results = query.fetch(1)
if results:
return _create_session_for_user(results[0])
raise Exception("Invalid username/password")
You'll need add an index for the above query to work. It seems like your not using app engine, so you probably have to add indices through the web console
answered Mar 9 at 0:59
AlexAlex
2,159415
2,159415
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
add a comment |
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
Thanks. It worked...locally. But turns out GAE only supports ndb in the standard env which seems better among the two anyway.
– Zain Qasmi
Mar 10 at 23:42
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55059285%2fparsing-datastore-entity-in-python-efficiently%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What are you trying to illustrate in your example? Your for-loop keeps overwriting the values in
result
with the next entity. You are effectively just doing this:result = query.fetch()[-1]
– Alex
Mar 8 at 16:52
ahh yes, in my case it would only ever have one entity which again I can't, or don't know, how to access directly. query.fetch() doesn't support indexing. I wouldn't be posting this question if it did.
– Zain Qasmi
Mar 9 at 0:16