Feature tools - temporal cutoffs do not register time index variable2019 Community Moderator Election“Large data” work flows using pandasPython, Pandas: Reindex/Slice DataFrame with duplicate Index valuescutoff time and training window at featuretoolsCreate features based on cutoff times in featuretoolsLookupError: Time index not found in dataframeRemove Rows Where the Person Has Not Changed LocationsFeaturetool deployment issueAre there built-in primitives for interactions in Feature tools?Automated feature generation for time series problems - FeaturetoolsFeature Tools default cutoff_time
I've given my players a lot of magic items. Is it reasonable for me to give them harder encounters?
Should I apply for my boss's promotion?
Why is there an extra space when I type "ls" on the Desktop?
Why restrict private health insurance?
Should I file my taxes? No income, unemployed, but paid 2k in student loan interest
What would be the most expensive material to an intergalactic society?
How to educate team mate to take screenshots for bugs with out unwanted stuff
Can multiple states demand income tax from an LLC?
std::string vs const std::string& vs std::string_view
What is Tony Stark injecting into himself in Iron Man 3?
Use Mercury as quenching liquid for swords?
What the error in writing this equation by latex?
Short story about cities being connected by a conveyor belt
After Brexit, will the EU recognize British passports that are valid for more than ten years?
Too soon for a plot twist?
Can I challenge the interviewer to give me a proper technical feedback?
Is there a logarithm base for which the logarithm becomes an identity function?
Why isn't P and P/poly trivially the same?
Is there a math expression equivalent to the conditional ternary operator?
Professor forcing me to attend a conference, I can't afford even with 50% funding
Does an unused member variable take up memory?
What is the best index strategy or query SELECT when performing a search/lookup BETWEEN IP address (IPv4 and IPv6) ranges?
Why does this boat have a landing pad? (SpaceX's GO Searcher) Any plans for propulsive capsule landings?
ESPP--any reason not to go all in?
Feature tools - temporal cutoffs do not register time index variable
2019 Community Moderator Election“Large data” work flows using pandasPython, Pandas: Reindex/Slice DataFrame with duplicate Index valuescutoff time and training window at featuretoolsCreate features based on cutoff times in featuretoolsLookupError: Time index not found in dataframeRemove Rows Where the Person Has Not Changed LocationsFeaturetool deployment issueAre there built-in primitives for interactions in Feature tools?Automated feature generation for time series problems - FeaturetoolsFeature Tools default cutoff_time
I am using feature tools to create monthly aggregations.
I have toy data consisting of loan applications (1000 ID_APPLICATION
; 1000 TIME_APPLICATION
)
and 200k transactions (-> ~200 transactions for 1 person; 1 transaction have information like AMOUNT
, TIME
and other, not needed for this example). TIME
column consists of ~200 different times for one person, in previous year or more.
constants.py
____________
ID_APPLICATION_COLUMN = "ID_APPLICATION"
ID_TRANSACTIONS_COLUMN = "ID_TRANSACTION"
TIME_COLUMN = "TIME"
TIME_APPLICATION_COLUMN = "TIME_APPLICATION"
ENTITY_SET_NAME = "clients"
TRANSACTIONS_ENTITY_NAME = "transactions"
APPLICATIONS_ENTITY_NAME = "applications"
creation
____________
# we fill the entity_set with the dataframes, and say, which IDs are relevant for given DF
entity_set.entity_from_dataframe(entity_id=cnst.TRANSACTIONS_ENTITY_NAME,
dataframe=transactions,
index=cnst.ID_TRANSACTIONS_COLUMN,
time_index=cnst.TIME_COLUMN)
entity_set.entity_from_dataframe(entity_id=cnst.APPLICATIONS_ENTITY_NAME,
dataframe=applications,
index=cnst.ID_APPLICATION_COLUMN,
time_index=cnst.TIME_APPLICATION_COLUMN)
# Specification of the relationship between entities
r_transactions_applications = ft.Relationship(
parent_variable=entity_set[cnst.APPLICATIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN],
child_variable=entity_set[cnst.TRANSACTIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN])
entity_set.add_relationship(r_transactions_applications)
However, I have problem with temporal cutoffs.
when I create them and apply them:
default_agg_primitives = ["count", "sum", "std", "max", "mode", "mean"]
default_trans_primitives = ['month', 'day', 'time_since_previous']
temporal_cutoffs = ft.make_temporal_cutoffs(
instance_ids=applications[cnst.ID_APPLICATION_COLUMN],
cutoffs=applications[cnst.TIME_APPLICATION_COLUMN],
window_size='1m',
num_windows=6)
transformed_data = ft.dfs(entityset=entity_set,
target_entity=cnst.APPLICATIONS_ENTITY_NAME,
cutoff_time=temporal_cutoffs,
cutoff_time_in_index=True,
trans_primitives=default_trans_primitives,
agg_primitives=default_agg_primitives,
max_depth=2)
As I am aggregating for the application level, I get 1000 rows without temporal cutoffs. What I get when I apply them is 6000 rows, however 5000 rows (all months before, except for the last one) are 0 or NaN, and the rest is just the same as if I would not be using temporal cutoffs at all.
To me, it seems that the TIME
column is not registered and the dataset is not splitted.
Where can I set this?
python feature-extraction featuretools
add a comment |
I am using feature tools to create monthly aggregations.
I have toy data consisting of loan applications (1000 ID_APPLICATION
; 1000 TIME_APPLICATION
)
and 200k transactions (-> ~200 transactions for 1 person; 1 transaction have information like AMOUNT
, TIME
and other, not needed for this example). TIME
column consists of ~200 different times for one person, in previous year or more.
constants.py
____________
ID_APPLICATION_COLUMN = "ID_APPLICATION"
ID_TRANSACTIONS_COLUMN = "ID_TRANSACTION"
TIME_COLUMN = "TIME"
TIME_APPLICATION_COLUMN = "TIME_APPLICATION"
ENTITY_SET_NAME = "clients"
TRANSACTIONS_ENTITY_NAME = "transactions"
APPLICATIONS_ENTITY_NAME = "applications"
creation
____________
# we fill the entity_set with the dataframes, and say, which IDs are relevant for given DF
entity_set.entity_from_dataframe(entity_id=cnst.TRANSACTIONS_ENTITY_NAME,
dataframe=transactions,
index=cnst.ID_TRANSACTIONS_COLUMN,
time_index=cnst.TIME_COLUMN)
entity_set.entity_from_dataframe(entity_id=cnst.APPLICATIONS_ENTITY_NAME,
dataframe=applications,
index=cnst.ID_APPLICATION_COLUMN,
time_index=cnst.TIME_APPLICATION_COLUMN)
# Specification of the relationship between entities
r_transactions_applications = ft.Relationship(
parent_variable=entity_set[cnst.APPLICATIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN],
child_variable=entity_set[cnst.TRANSACTIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN])
entity_set.add_relationship(r_transactions_applications)
However, I have problem with temporal cutoffs.
when I create them and apply them:
default_agg_primitives = ["count", "sum", "std", "max", "mode", "mean"]
default_trans_primitives = ['month', 'day', 'time_since_previous']
temporal_cutoffs = ft.make_temporal_cutoffs(
instance_ids=applications[cnst.ID_APPLICATION_COLUMN],
cutoffs=applications[cnst.TIME_APPLICATION_COLUMN],
window_size='1m',
num_windows=6)
transformed_data = ft.dfs(entityset=entity_set,
target_entity=cnst.APPLICATIONS_ENTITY_NAME,
cutoff_time=temporal_cutoffs,
cutoff_time_in_index=True,
trans_primitives=default_trans_primitives,
agg_primitives=default_agg_primitives,
max_depth=2)
As I am aggregating for the application level, I get 1000 rows without temporal cutoffs. What I get when I apply them is 6000 rows, however 5000 rows (all months before, except for the last one) are 0 or NaN, and the rest is just the same as if I would not be using temporal cutoffs at all.
To me, it seems that the TIME
column is not registered and the dataset is not splitted.
Where can I set this?
python feature-extraction featuretools
add a comment |
I am using feature tools to create monthly aggregations.
I have toy data consisting of loan applications (1000 ID_APPLICATION
; 1000 TIME_APPLICATION
)
and 200k transactions (-> ~200 transactions for 1 person; 1 transaction have information like AMOUNT
, TIME
and other, not needed for this example). TIME
column consists of ~200 different times for one person, in previous year or more.
constants.py
____________
ID_APPLICATION_COLUMN = "ID_APPLICATION"
ID_TRANSACTIONS_COLUMN = "ID_TRANSACTION"
TIME_COLUMN = "TIME"
TIME_APPLICATION_COLUMN = "TIME_APPLICATION"
ENTITY_SET_NAME = "clients"
TRANSACTIONS_ENTITY_NAME = "transactions"
APPLICATIONS_ENTITY_NAME = "applications"
creation
____________
# we fill the entity_set with the dataframes, and say, which IDs are relevant for given DF
entity_set.entity_from_dataframe(entity_id=cnst.TRANSACTIONS_ENTITY_NAME,
dataframe=transactions,
index=cnst.ID_TRANSACTIONS_COLUMN,
time_index=cnst.TIME_COLUMN)
entity_set.entity_from_dataframe(entity_id=cnst.APPLICATIONS_ENTITY_NAME,
dataframe=applications,
index=cnst.ID_APPLICATION_COLUMN,
time_index=cnst.TIME_APPLICATION_COLUMN)
# Specification of the relationship between entities
r_transactions_applications = ft.Relationship(
parent_variable=entity_set[cnst.APPLICATIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN],
child_variable=entity_set[cnst.TRANSACTIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN])
entity_set.add_relationship(r_transactions_applications)
However, I have problem with temporal cutoffs.
when I create them and apply them:
default_agg_primitives = ["count", "sum", "std", "max", "mode", "mean"]
default_trans_primitives = ['month', 'day', 'time_since_previous']
temporal_cutoffs = ft.make_temporal_cutoffs(
instance_ids=applications[cnst.ID_APPLICATION_COLUMN],
cutoffs=applications[cnst.TIME_APPLICATION_COLUMN],
window_size='1m',
num_windows=6)
transformed_data = ft.dfs(entityset=entity_set,
target_entity=cnst.APPLICATIONS_ENTITY_NAME,
cutoff_time=temporal_cutoffs,
cutoff_time_in_index=True,
trans_primitives=default_trans_primitives,
agg_primitives=default_agg_primitives,
max_depth=2)
As I am aggregating for the application level, I get 1000 rows without temporal cutoffs. What I get when I apply them is 6000 rows, however 5000 rows (all months before, except for the last one) are 0 or NaN, and the rest is just the same as if I would not be using temporal cutoffs at all.
To me, it seems that the TIME
column is not registered and the dataset is not splitted.
Where can I set this?
python feature-extraction featuretools
I am using feature tools to create monthly aggregations.
I have toy data consisting of loan applications (1000 ID_APPLICATION
; 1000 TIME_APPLICATION
)
and 200k transactions (-> ~200 transactions for 1 person; 1 transaction have information like AMOUNT
, TIME
and other, not needed for this example). TIME
column consists of ~200 different times for one person, in previous year or more.
constants.py
____________
ID_APPLICATION_COLUMN = "ID_APPLICATION"
ID_TRANSACTIONS_COLUMN = "ID_TRANSACTION"
TIME_COLUMN = "TIME"
TIME_APPLICATION_COLUMN = "TIME_APPLICATION"
ENTITY_SET_NAME = "clients"
TRANSACTIONS_ENTITY_NAME = "transactions"
APPLICATIONS_ENTITY_NAME = "applications"
creation
____________
# we fill the entity_set with the dataframes, and say, which IDs are relevant for given DF
entity_set.entity_from_dataframe(entity_id=cnst.TRANSACTIONS_ENTITY_NAME,
dataframe=transactions,
index=cnst.ID_TRANSACTIONS_COLUMN,
time_index=cnst.TIME_COLUMN)
entity_set.entity_from_dataframe(entity_id=cnst.APPLICATIONS_ENTITY_NAME,
dataframe=applications,
index=cnst.ID_APPLICATION_COLUMN,
time_index=cnst.TIME_APPLICATION_COLUMN)
# Specification of the relationship between entities
r_transactions_applications = ft.Relationship(
parent_variable=entity_set[cnst.APPLICATIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN],
child_variable=entity_set[cnst.TRANSACTIONS_ENTITY_NAME][cnst.ID_APPLICATION_COLUMN])
entity_set.add_relationship(r_transactions_applications)
However, I have problem with temporal cutoffs.
when I create them and apply them:
default_agg_primitives = ["count", "sum", "std", "max", "mode", "mean"]
default_trans_primitives = ['month', 'day', 'time_since_previous']
temporal_cutoffs = ft.make_temporal_cutoffs(
instance_ids=applications[cnst.ID_APPLICATION_COLUMN],
cutoffs=applications[cnst.TIME_APPLICATION_COLUMN],
window_size='1m',
num_windows=6)
transformed_data = ft.dfs(entityset=entity_set,
target_entity=cnst.APPLICATIONS_ENTITY_NAME,
cutoff_time=temporal_cutoffs,
cutoff_time_in_index=True,
trans_primitives=default_trans_primitives,
agg_primitives=default_agg_primitives,
max_depth=2)
As I am aggregating for the application level, I get 1000 rows without temporal cutoffs. What I get when I apply them is 6000 rows, however 5000 rows (all months before, except for the last one) are 0 or NaN, and the rest is just the same as if I would not be using temporal cutoffs at all.
To me, it seems that the TIME
column is not registered and the dataset is not splitted.
Where can I set this?
python feature-extraction featuretools
python feature-extraction featuretools
asked 2 days ago
johnnyheinekenjohnnyheineken
156114
156114
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55023395%2ffeature-tools-temporal-cutoffs-do-not-register-time-index-variable%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55023395%2ffeature-tools-temporal-cutoffs-do-not-register-time-index-variable%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown