Various neural networks were looked into and we tried to implement them. The main method being a LSTM, however again resources became the bane of our existence as running the models on GPU quickly overcame the TITAN’s VRAM while running on the CPU was estimated to take multiple days in which we just did notContinue reading “PyTorch & Keras”
Author Archives: thestallions
Interpolation
The interpolation code take a very very long time to run even on the demo datasets, however using the demo datasets may have cost us some time in the long run here because after using the interpolation on the full datasets there were some more issues discovered, that there were some patients with 0 dataContinue reading “Interpolation”
The Initial Observation
After evaluating the LSTM model in Keras, it was seen that a low Mean Squared Error was given along with a terrible accuracy and F1 score. This meant that the time series data was either highly imbalanced or the model was not properly configured. We decided to try creating an LSTM model in PyTorch instead.
Interpolation!
We spent a good amount of time testing the interpolation cod eon the sample dataset. The code takes a while to run so I’m worried it will take a while on the full data. But it works so that’s good. We also got word from our supervisor that we would be able to get accessContinue reading “Interpolation!”
The Interpolation Experiment
After successfully merging the data files together, data interpolation had to be done. This meant that it was necessary to fill the missing values with values that were last obtained for a patient. This can be seen in the diagram below. This brought the data into a time series format. Model development was also doneContinue reading “The Interpolation Experiment”
Data Merging
Much of the last two weeks was spent, in sorting and merging the data into patient time-series format. After some deliberation with our supervisor the it was decided that the empty cells would be filled using the nearest value time-series wise.
Collision Course
This week we merged the separate cleaned files together into one the sample data file is starting to get big at this point and the merge was interesting to perform. We settled on performing an outer join so that information isn’t lost since we ensured we are only keeping information for patients that have aContinue reading “Collision Course”
All Tidy
Not much has happened this week just cleaned the last bit of the files that needed cleaning. The next step is to merge the data together hopefully it goes smoothly since even the sample data is pretty large. Nothing much else to report on for now things are getting exiting though soon we will beContinue reading “All Tidy”
The Merging Configuration
This week consisted of us generating Jupyter notebooks in order to efficiently merge and produce a combined data set with patient data. This involved us studying the best methods of joining tables and merging in pandas and by examining the data’s offset values. These offset values were all synced together in order to present timeContinue reading “The Merging Configuration”
The Documentation Formulation
In this week, we began the write up for documentation. This is a very lengthy process and will continue to go on for weeks to come. Mainly only the known sections were done and the document was laid out. Hopefully it isn’t hard to complete in the weeks to come.
The Stallions Chronicle 