Ufdjrw

I have a dataframe that contains a pre-processed data, such that every 4 rows is a sequence (later to be reshaped and used for lstm training).

I want to shuffle the dataframe, but I want to keep every sequence of rows untouched . For example:
a = [1,2,3,4,10,11,12,13,20,21,22,23] will turn into something like: a = [20,21,22,23,1,2,3,4,10,11,12,13].

df.sample(frac=1) is not enough since it will break the sequences.

Solution , thanks to @Wen-Ben:

seq_length = 4 
length_array = np.arange((df.shape[0]//seq_length)*seq_length)
trunc_data = df.head((df.shape[0]//seq_length)*seq_length)
d = x : y for x, y in trunc_data.groupby(length_array//seq_length)
yourdf = pd.concat([d.get(x) for x in np.random.choice(len(d),len(d.keys()),replace=False)])

Is this what you need , np.random.choice

d=x : y for x, y in df.groupby(np.arange(len(df))//4)

yourdf=pd.concat([d.get(x) for x in np.random.choice(len(d),2,replace=False)])
yourdf
Out[986]: 
 col1 col2
4 5 e
5 6 f
6 7 g
7 8 h
0 1 a
1 2 b
2 3 c
3 4 d

You can reshuffle in groups of 4 by... grouping the index into groups of four and then shuffling them.

Example:

df = pd.DataFrame(np.random.randint(10, size=(12, 2)))

 a b
0 5 4
1 7 7
2 7 8
3 8 4
4 9 4
5 9 0
6 1 5
7 4 1
8 0 1
9 5 6
10 1 3
11 9 2

new_index = np.array(df.index).reshape(-1, 4)
np.random.shuffle(new_index) # shuffles array in-place
df = df.loc[new_index.reshape(-1)]

 a b
8 0 1
9 5 6
10 1 3
11 9 2
4 9 4
5 9 0
6 1 5
7 4 1
0 5 4
1 7 7
2 7 8
3 8 4

As you said that you have the data in sequences of 4, then the length of the data frame should be in multiples of 4. If your data is in sequences of 3, kindly, change 4 to 3 in the code.

>>> import pandas as pd
>>> import numpy as np

Creating the table:

>>> df = pd.DataFrame('col1':[1,2,3,4,5,6,7,8],'col2':['a','b','c','d','e','f','g','h'])
>>> df
 col1 col2
0 1 a
1 2 b
2 3 c
3 4 d
4 5 e
5 6 f
6 7 g
7 8 h
>>> df.shape[0]
8

Creating the list for shuffling:

>>> np_range = np.arange(0,df.shape[0])
>>> np_range
array([0, 1, 2, 3, 4, 5, 6, 7])

Reshaping and shuffling:

>>> np_range1 = np.reshape(np_range,(df.shape[0]/4,4))
>>> np_range1
array([[0, 1, 2, 3],
 [4, 5, 6, 7]])
>>> np.random.shuffle(np_range1)
>>> np_range1
array([[4, 5, 6, 7],
 [0, 1, 2, 3]])
>>> np_range2 = np.reshape(np_range1,(df.shape[0],))
>>> np_range2
array([4, 5, 6, 7, 0, 1, 2, 3])

Selecting the data:

>>> new_df = df.loc[np_range2]
>>> new_df
 col1 col2
4 5 e
5 6 f
6 7 g
7 8 h
0 1 a
1 2 b
2 3 c
3 4 d

I hope this helps! Thank you!