Posts

Showing posts with the label dataframe

R: How to find the first non-zero element in a datframe by group

I have the following dataframe I want to subset this dataset such that I will only have rows between the first record and the the first 1 value in the flag column for each group and if there is no 1, that group should not appear at all. Something like this: I saw some answers at Dplyr : how to find the first-non missing string by groups? But it is for non-missing and I have both non-missing and 0 values. Or in dplyr (same result) Data used: Benchmark Output: Benchmark code: Using dplyr::slice , then the equivalent base R using by . And finally one just for performance, with benchmark. All robust for the case with no Flag==1 in a group. dplyr base base fast benchmark I did the benchmark on @Lebatsnok modified input, that I remodified because the NAs were not properly recognized as such. MKR's and WWW's solutions are not robust for this case, but i left them in the benchmark anyway. data benchmark code A cumsum based solution using dplyr can be as: Data: A solution using dply...

Process rows of R dataframe without loop in memory efficient way

The structure of my dataframe data1 , which has over 1.5 million rows, is like this: I need to insert a column Exit.time using values in columns WEEK and END and a cutoff value, which is 1287. Exit.time should have 0 or 1 value based on the following logic: if WEEK = 1287, then Exit.time = 0. if Week not equal to 1287, but WEEK = END then Exit.time = 1, otherwise Exit.time = 0. For this I tried the following for loop and it does what is required in the above dummy data set. The problem is that when I use the above loop in my real data set, even after an hour I am not getting an output. I guess looping is not efficient given the size of the dataset. Is there an alternative way to do what I want? I prefer to maintain the order of rows in data1 since I need to do some merge operations later on. Since you need Exit.time to be 1 when (WEEK == END) & WEEK != 1287 and 0 otherwise, you can use as.numeric on the results of (WEEK == END) & WEEK != 1287 , which changes TRUE...