Yeah, that'll be awesome. So yeah, and if you have any questions, feel free to reach out to us or comment on our discussion.
As well. We also wanted to let you know that this current challenge, the one that you just were involved in, the paper is currently being drafted in the tentative title is putting computational models of the to the test and invite a challenge to predict the protesters vaccination outcomes.
So, and in the team is currently reviewing model submitted by the contestants. We would love for you all to be involved in credit as co-authors and we'll send out more information in the coming weeks.
So if you can look out for that, that'll be awesome. And then this next third public challenge that we'll be having, and we want to let you know that we'll be announcing this in August of this year, 2024.
And we would love for you all to be involved. So if you can register, if you go to the CMIPP website, we'll have all that information there as well.
Cool. And I know we're super, short on time with the amount of awesome presentations that we have.
So I'll stop right there for the introductions and I'll go ahead and pass that off to
I will share my screen.
We can see it well, yes. Thank you.
Yeah. Yeah.
Can you see it? Right. Well, first I'd like to thank the digital challenge organizers for arranging this, project.
We, you know, besides the compelling goals you have for doing that. Data challenge at the University of Minnesota, it just brought the faculty and students together to participate in this data challenge.
Worked as a team and I would like to introduce my other team members were Michael Andersen, Cheng Cheng Ooo, Josie Sorenson, Catherine Lee and Vargo Kakuti.
I should say that they were really the main drivers for this data challenge and worked really hard to get this done.
I'm really proud that we, the performance and the end result so good us to the team. So I'll go over, you know, really briefly what we did.
So a brief introduction of the data. So we had training data and test data. The training data had 2 different years of data.
We had same frequency, multomic data on individuals. So we had cell frequency, gene expression, cytokine and plasma antibiotic data on individuals using this 2020 2021 datasets we were supposed to predict the ranks using the baseline redouts, we were supposed to predict the ranks using the baseline redouts for the 22 data sets using the baseline redoubt for the 22 data set which has only the
baseline redoubt for the 22 data set which has only the baseline data.
So just making sure, only are you advancing the sites because we still see the title slide.
Oh really? Okay.
Yes.
This looks, yeah, yeah, this works better.
Okay. Alright, so then we were, we had demographic data on individuals as well. So what we did initially.
You've looked at the demographic data and wanted to make sure and I should mention there are 2 vaccine groups.
So we wanted to make sure that the demographic distributions, the different factors of the variables were not.
Significantly differentially distributed between the 2 vaccine groups and as well as between the training and the test data.
So when I looked at it, you know, there were some, the age was a little bit differential as you can see within the vaccine groups but they were very similar between the training and the test data.
That there is a factor there were some differential distribution between the training and the test data set. So when we built our model, we made sure again all the races were included in our training data set.
Training model. So given that then we, and one big component of our analysis was the pre-processing steps.
I mean, it's, it's, multimodal Linux data and as you know, the pre-processing steps are really important.
So this is a little busy slide but as you can see the different modalities of the data we have here the mix data and then when you did the data harmonization we keep the common features for its modality and then we saw a pretty significant batch effects and I will show you some graphs on that.
So we did batch correction. We did normalization and then we ended up including there are some serious missing the gene feature so we excluded them if they are 70% they were missing the observations were missing so we excluded them.
And again, when we harmonized, we keep the common features. So these are the number of features that were left for each modality and we combined these datasets and then we have multiple time points available on each of these data sets.
So again, we looked at all the time points and all the years of data. We imputed the missing observations that will show you the slides on that and we did time-wise imputation.
And then again we separated the training data and the test data and build our model. Did the analysis.
So just as I mentioned this, batch effects as you look at the first row is the PC plots and trying to show the batch effects.
For the different data set if you look at the RNC data and cytokine you can see pretty significant batch effects there.
So we had to correct for that and that is see if you don't do it, of course that will play a big role in your prediction.
So we, we did this batch correction and then the corrected data looked pretty good. So we, move forward with this, harmonized, connected data sets.
And then we did imputation, we used the Bayesian PC approach. To do the missing data imputation it was getting pretty computational intensive so actually we did imputation at each time point separately.
And then the rate and the black one showing you the imputed, the non- of the observed data.
And again, we made sure, you know, after imputation, the data doesn't look, very differently.
Distributed as opposed to the observed data. So at the end of the imputation, harmonization, normalization, we for each of these modalities we have this many features left.
And, like I said, we, for the training data, these are the 2 data said 2 years we combined.
They looked pretty nicely, uniformly distributed and we built our model based on that. Michael, go ahead.
Yeah, thank you. So just a quick reminder of the different tasks. The first one involving the ante body was split into 2 parts.
We want to build a model that's capable of giving ranks based on the absolute value of the protussis toxin antibody but 14 days after boosting as well as full change from the baseline at day 0.
Next slide, please. Thanks. And so then the cell frequencies task is similar in that we want to be able to predict mono sites, and rank individuals based on mono site absolute value one day after boosting as well as full change one day after boosting.
And then finally the CCL 3 gene was selected as our biomarker for gene expression.
And we wanted to be able to rank individuals and do full change 3 days after boosting.
The model that we ultimately decided to use. Was based on the different modalities. So the the cell frequency cytokine antibody.
And. Gene expression data. And And we opted to use multiple call inertial analysis and we generated 10.
Predictor vectors and we use these to generate a linear mixed effects model which I will go into the coefficients and details shortly.
And we didn't just decide to do the MCIA model. We also tested Lasso, Ridge, partial least squares and principal components analysis.
The way we settled on the MCI model was we combined the 2020 and 2021 data as our training data set.
And we would split it. Roughly say, probably like one fifth, 4 fifths training and testing within that.
We kept the one fifth as our testing. Validation set and we would train one of the models.
So lasso or bridge or PCA. We use 5 fold cross validation to train the models.
And then we use the withheld testing set. From the 2020 and 2021 data. To validate the models of which ones seem to be the most efficient.
And we settled on MCIA as being the most robust and efficient model to be used. With the linear mixed effects.
And so then what we did was we took the 2020 and 2021 data combined. We built a model using MCIA and the linear mixed effects model.
And then we trained our 2022 data from the baseline values.
And the model that we used involved 10 different MCIA components. What we did was we separated the data.
Based on time 0, time one, time 3 times 7, time 14. We took all the modalities together for a particular time point and we would construct 10 components from the MCIA model and use those as a dimension reduced coefficients in our linear mixed effects model.
For our mixed effects model, we opted to use subject ID as our random effect because the different observations longitudinally are not independent of one another.
They are clustered based on the subject. So we included that as our random effect. And then the remaining 10 MCI components were fixed effects that we used as beta coefficients.
To handle the time point potential time point differences we included time as a fixed effect. As well as interactions as fixed effects between the first 3 MCA components against time.
And so that was how we were able to. Construct our linear mixed effects model.
And the mathematics behind solving. This was essentially just getting best linear unbiased predictors for all the different mix effects.
And we were able to use the estimated beta coefficients to predict the absolute value. Sort of monocyte cell frequency antibody.
Values. For the predict or for the withheld 2022 data.
And our results suggested that for the antibody both the absolute value and the rank or sort of full change our spearmint correlations show that it was pretty.
Pretty reasonable estimates. Both are monocyte and our CCL 3 absolute value expression levels were pretty reasonable, but our full change was a little bit less than satisfactory and that could be due to potential normality assumptions within the linear mixed effects model.
Well, thank you.
Thanks very much.
Thank you both. Does anyone have any quick questions?
It's so cool. I love it. It's awesome to see it and There's lots of questions and we don't have the time here, right?
I would love to talk about some of the details and I'm sure others did. It's nice to see how the I mean, multiple components of what you did were used by others as well, but I think you did like, I mean, my first guess is I think your pre processing and normalization is key here to what you.
So the success, but, yeah, anyway, sorry.
Thanks again. I'll put a link in the chat. For the discourse, if you have any questions, if you can post them there, that'll be awesome.
Okay. But that, Nikki, if you can go up next, that'll be fantastic.
Oh, hi everyone. Took me a while to bring this off. Can you see my screen?
Yeah, looks good.
Okay. Excellent, thank you. Yeah, indeed. Thank you, to the CMI PB team for a really interesting challenge.
And well done to the University of Minnesota. It was really interesting to see how you put your model together.
So I'm gonna share with you my model. So I really took, a data integration approach.
To the modeling. So just to give you kind of an idea of the concept behind model.
So we know with kind of multi omix measures that they tend to be a little bit, there can be some random measurements in there.
And we also know there can be some quite technical biases. Which the previous team pointed out already.
And also we want to see if we can really maximize different signals coming from multiple sources, right? And so we want to see if we could address this with a data integration approach.
And so really, the main idea is to see if we can leverage different measures that are similar across modalities at the same time.
We also hope to be able to remove some of the noise in the data. And another added bonus is that it's biologically interpretable.
So the package that we used with MOFA, so it's multi omix factor analysis for data integration.
And very basically it's a it's a matrix factorization approach that takes all your different modalities and it projects them into a load-dimensional space.
So at the end you have. A factors by sample matrix. And then you can, you can look at the different factors and try and understand.
Based on their variance where your signals are coming from and which which factors are potentially interesting.
In terms of the results. So We tried this 2 different ways. We had 2 different models.
The first idea was to train on the 2,020 data and then test on the 2,021 data.
And then eventually try and predict the 2022 data. And then the second approach we took the 2020 and 2021 data and we split it into a test train.
I think the split was 80 20 if I remember correctly. And so we used the latter regression approach.
And that prediction features were, our baseline values and our demographic information. And so that's the baseline model that you can see these gray bars representing here.
And then to that baseline model, we, we added the top moder features.
And that's what you can see in these turquoise bars here. And then finally we applied the model to the prediction task for 2022.
And that's what you see in the blue. And the start, so we use the correlation as a matrix, so the same as the CMI PB team did.
And we can see that the mofer, factors made some tiny improvements on the baseline model.
But for especially for CCL 3, it seemed very predictive. So it formed really well in the actual task.
And then also surprisingly, the IGG, PC fault change, it performed pretty well as well.
And then for the mono sites, for the absolute value of the monocyte prediction, it Yeah, performed well.
So in conclusion, yeah, the models predicted CCL expression well, and also the IGG PT fold change.
The most factors did seem to have a, and yeah, have a good effect on the models.
There was still some word to do on the, on the motor model itself, so I think we could probably improve the result even further.
And it was also really interesting to note that There were very similar results between the 2020 and the 2021.
The 2020 training and the 2020 plus 2021 training. So in this graph I've just showed.
The results from the 2020 plus 2021 training but the results from the 2,020.
We're very similar.
Yeah, so that's it. Thank you very much.
Thank you, Nicki. Also put, the link to her discussion thread in the chat as well.
Thank you.
Okay, awesome. Thank you. If whenever you're ready, if you can start screen sharing, that'll be awesome.
Hi.
I'm good.
Yeah, sounds good.
Thank you.
Good. Hey, I'm good. Then, but, but, then, but, but, then, but, but then, because, you know, at the time, I'm trying to make that And, I'll do that.
I, But, So, Good. Only, No, So, What do you mean?
So, the Yeah.
Hello. That the And I And, the. At the, the, 20 or 20, that said at that, At Thank you, Good.
That's it. And When I train early or 13 to 10, I said, the. I like.
That Good It's Hmm, The yeah. One to 31 to 3, 3, At the at the Hello, Good.
We got today. I think energy. I, I think,
And, At I do it. That, and that.
Awesome, thank you, Mahita. And I will also post. Meet us link in the chat as well.
Thank you.
Awesome. Thank you.
Okay, next we have William. If you're able to screen share.
Yes.
Okay, there you go.
Cool. So yeah, I took kind of a different approach. To this whole challenge, my big interest was trying to see what I can learn from each individual.
Relative to the rest of the test population. So, in total, I just ignored the, That gene expression data just to simplify things for myself and went with the cell frequencies, cytokines and plasma titers, just using the pre-processed data across both data sets.
So what I did was for each subject I would create a random forest classifier where the output was.
Or where the model was trained on every other individual across the datasets. And the output was classifying whether the input individual was ranked higher or lower than the random forest that was based off of that one subject.
So what I ended up getting is that for each subject, I had a random forest model that was used to rank every other individual in this study.
And I just thought this was an interesting approach because I could then go into each individual and see, okay, how are their trees constructed, because I could then go into each individual and see, okay, how, how are their trees constructed and what features are they using to rank other people relative to themselves?
So from here. I then combined the prediction. So it would be either a, you know, input individual either ranked higher or lower using just their baseline features.
And then I would combine that with, the trees, self predicted probability, which is just based on the number of votes that each feature got in those trees so and i played around with the scaling probability for how much that impacted.
I guess how important the vote was for each model. So when I ended up with was a matrix where for each input subject each model would have a vote whether it's ranked higher or lower than that individual and then that was scaled by The self-predictive probability of that tree.
Now from here, my thought was, okay, I can use, XG boost, or, or another algorithm just to try and figure out.
Which individuals were good predictors and which were bad because my hypothesis is that some individuals are good at predicting How to rank other individuals that are in kind of a similar class of response.
But overall there is not going to be a single shared modality. So sometimes a individual model be able to predict really accurately whether someone's higher or lower.
And sometimes, that individual model would predict really poorly. So after, kind of, using actually boost to, look at my matrix and kind of.
Spit out a prediction. Base off kind of the quorum of my models.
I got a kind of kind of middle, like load to decent spearman correlations for the 4 tests or the 3 test I attempted.
Which is the IGGs and the The monocytes. Now for the monocyte full change, that was really interesting because I get a really strongly negative spearmen correlation.
So my model is doing a really good job. I predicted the exact opposite of who is ranked low.
So, afterwards I went back and revisited. How my ex-te boost was looking at my matrix and basically it was taking all the really bad predictors and valuing them really highly and all the good predictors were being valued really low so it was kind of yeah, it, it was, pulling a nice little, you know, Uno swap on me.
But I think overall, just kind of poking around in each of these individual trees I learned a lot about how individuals were responding differently and I think going forward this kind of gives me more ideas of you know, how there's, you know, maybe not a single modality or maybe there is kind of multiple ways that people can be responding, but how you can kind of use each of these individual responses to.
yeah, build towards a, prediction based off of what, what your input is.
Kinda depending on your different response groups but yeah That was That was it for me. So if you know what I ask questions, feel free to.
Send me a chat. Thank you.
Thank you. Thank you so much. Okay, great. And I will.
And interest of time I'll just go ahead and share my screen. Okay, great.
So thank you everyone for all your. The presentations and if you have any questions again, please use the links in the chat.
We also wanted to let you know. Have we had specific questions that we had. Through the feedback poll that you all kindly, gave us input on.
So I wanted to summarize that quickly here. So, the first question or one of the questions that we asked was, how helpful were the resources and tools provided, during the challenge.
What people said was the most, useful were the direct communications, and the same IPv website was also useful.
While Zoom information sessions, were maybe not as useful and we'll have a question, later in the session, but, just to summarize that here.
And then we also wanted to know how you would rate the difficulty level of the tasks, one being very difficult and 6 being very simple.
It was averaging around 3.2 2. So that was our goal and doing that. So that's great to hear wasn't.
Too crazy hard or too crazy easy. We also asked how clear the information was, on the CMIBB website.
And the average was a 4.9 4 with, one being fair and clear. So, definitely room for improvement, which is why we're having this session and we'd love to hear your feedback.
So yeah. And then next, which is really great to hear. So we asked how likely if we were you to participate in a similar challenge in the future.
So, people reported that they would be very likely to participate again so people that presented and those that you see on the call, you probably see, involved again.
So if you wanted to collaborate with each other, discuss each other there on the discussion form, I think that'd be a great idea.
And then, we also asked at the very end how people would rate their overall experience with the prediction challenge and people overall said that they were very satisfied, which is great to hear.
And so we really appreciate, everything that you've done, throughout the second challenge.
So with that, we had a couple of questions here. So the first one we had, we wanted to know how we can make you know, the Zoom sessions better.
We got feedback maybe we can make the zoom sessions more available by having them live on YouTube to increase accessibility.
We thought of maybe even having you know, maybe another zoom session where it's just purely updates and QA.
And we also thought maybe it'd be useful to have an open office hour if you ever, you know, had a professor that would have open office hours.
Where you can ask any questions about your particular model and stuff? But yeah, does anyone have any particular feedback or comments about the Zoom?
So, sorry, Mary, to interrupt, but like, and I think I should reflect on like a minute introduction here.
So for, this is the first time we did this. With people who are not part of one of our grants and stuff so and we do wanna continue doing it, right?
Okay.
So the reason why we asking these questions is primarily because we want to do a better drop going forward. I thought as Mari was presenting it was already pretty good Obviously we can improve.
And there were some things we did terribly from. I hope we. Yeah, we, address that most of those.
So but here the reason why this is our last kind of invited challenge where we half is small.
Number of people you guys here on the call. And we are planning for another one in August. Where then we would have Hopefully a hundred people or so 100 teams joining And then there's gonna be much harder to do some of the things like the direct feedback, right?
It's not gonna work. We can't scale it up. And there was overall like, I think massive success here in terms of like, could be, the, the fact, the very different models that made first and Williams really unique approach, I thought that was really, really cool.
Anyway, so I guess like that's why we're asking. And that would.
Love any kind of feedback. On what do you think, how we can do better and Mary was pointing out here like the zoom being the first one.
Yeah, if you could unmute yourself and try.
Okay.
I can get a started. On this front. I guess my biggest advice. For the beginning is the solution center is so helpful and maybe at the beginning when somebody submits direct feedback and you
You can in the first Zoom Meeting. Just tell them here is, you know, Primod literally taking this question and putting it on the solution center and what he does that he sees that oh another person already asked this question and just going through that process I think it will both enable people to use that resource very effectively early on and make a big difference.
Thanks, Todd. That's really helpful.
Anything else? And if not, we can go for it because I know we have a Couple more questions here.
Okay, yeah, if anything comes up later, if you can email me or put it in the discussion forum, that'll be awesome.
Cool. There are a couple, you know, on the pre response section, people commenting about the valuation metrics.
So we wanted to bring it up here. So for the second challenge, we had a point system.
As you might be very well aware. So for a completion of a task we gave, the participants 1 point and then the winner of each task got 3 points.
So with that and kind of getting the feedback and hearing from other people. We had also wanted to propose a revised point system for our next public challenge.
So you have 2 options here. So option one, the winner would get 4 points and then a 98% confidence interval of the winner would also get 3 points and then if the result was significant 2 points and then if they submitted the task they get 1 point.
So the maximum per task for option one would be 4. We also got feedback that maybe the top 2 rows here maybe should have more weight to it.
So we would multiply that by too. So the winner would get 8 points. And then the second line item, of 3 would get 6 points and this was the total maximum per task.
With option 2 would be 8. So we wanted to open it up. Do you have any thoughts, comments?
Did you like how it was this last time or? Do you have any thoughts about option one or 2?
Hmm.
I have a question. So there were definitely tasks that were harder or than others. Like there were tasks that nobody got right and they were tasked.
Hmm.
Everybody got right. Will the tasks have or have you thought about the tasks having different weights based on how hard they are well not based on how hard they are.
Is that should be based on how many people got got it right.
So what I'm asking is the different ways for different tasks. Because they, some task can be much harder.
Provide do you want to take this one? Okay.
Yeah, sure. I think it's it's really difficult. I like first challenge we just just to give you a big, the background on like how we selected task.
So in the first cell, we started with the 14 task and then there was a question like which task to be.
Put forward and then we picked which are not very obvious but which can be predicted well and for which we have like good data to represent those tasks and those predictors.
So we ended up selecting top 3 task. Not top 3 like the best 3 task in a way we understand the purchases by biology and then there are another kind of nice Nice, suggestion was that can we include full changes instead of directly asking the what happens at the endpoints and that's why we ended up like adding full changes and maybe that led to have like some task like really complex to predict and some are
not in different ways. Yeah, so this is the setup we have right now and it is good to hear about like yeah like waiting system according to the task predictions but yeah we need to think about this but yeah.
Yeah, we don't have like quick answer at the moment.
Yeah, that's an interesting point.
Yeah.
I think the idea of having. Like points distributed across, you know, in that case where there are a lot of.
People that are doing really well at the task. Versus ones that Yeah, like no one gets right or like one team is able to get right.
I think that Can be a good idea, but it's also just difficult to see which tasks are going to be harder at the onset.
I'm not exactly sure how you evaluate that, but, but I think in general though having Kind of a stronger waiting.
So like this option 2. Might be better. Though I did see like a couple of the top teams were all very close in terms of.
They're spear in correlations. So that was cool to see that a lot of models were.
Very close and it would be nice to see Yeah, teams getting rewarded for being second place in the category or something.
Yes, I can.
Awesome, thank you. Anything else and thanks for, putting your thoughts into the chat as well.
So it sounds like option 2. Is more popular here.
I would. I'm sorry. No, go for it too.
One. Go for it, Julie. I'll be quick. Just one quick thought as well and kudos again to you guys for coming up with the point scheme.
It's really difficult to quantify these things. One thing that we are doing as well is that we're summing across all the tasks, right, where we're not really celebrating potentially if one model did well biologically on on like the CCL, on like the gene expression data, maybe it could also be an idea to separate different categories and then that would kind of take care of the difficulty task
metric because I thought that comment was very valuable. So just one thought as well.
Thanks, Jeremy. And Julie, what were you going to say?
Kind of the same actually and I was just wondering whether yeah there is like an overall winner that does well in all the tasks but Maybe one does a team that's particularly well on only one task, but not so good in the remaining.
So in the end, they don't get to win, but should they get rewarded? For having done really well.
At one test.
Awesome.
Yeah, yeah, yeah, and then maybe actually also like. Like the reason why we ask you guys this is because when we wrote the whole thing we thought okay as long as we have a metric of what is a good and be all good, right?
And then we realized we have a really hard time of now comparing and people do want to know. It's unfortunately it's a very human thing to wanna have rankings and stuff, right?
And so. Yeah, and it is important. So yeah, that's why we're doing this and, this makes a lot of sense and then Yeah.
Thank you for commenting.
Thanks. Okay, we'll move on to the next question. For now. So our next one, we wanted to also get your thoughts on how we can engage more people for our third and public challenge.
So for this round, we went around and identified researchers from different scientific publications, and set them in, invitations individually.
One of the key issues that we saw when people, would come back to us individually, you know, people said they would be interested but didn't have the resources, you know, personnel, time to participate, which seemed like the main, you know, email that we would get.
But the next round we were hoping to reach out to the same people. Maybe we'd do more reach out through social media.
Promote is going to different conferences to also engage more people, but does anyone have any ideas or thoughts on how we can, you know.
Have people get involved if you, you know, What helped you get engaged and involved in what was, well, you know, interesting to you.
I can start this off. So I think, one thing that, What one thing about like separating out the competitions and maybe like the easy medium and hard tier to kind of put in perspective is, you know, telling people it's okay to participate in one tier and kind of giving people like a.
Like a low level in, a low level of engagement necessary. Test or task because I found that you know like I was just a person doing this you know for fun in my free time and I was originally just looking at the IGGs.
But as I got into it, I was like, oh, well, I want to do more and more.
So. I think the idea of kind of you know, maybe laying the breadcrumbs of giving some people multiple.
Tasks that have like different prize layouts could entice people if that kinda Makes sense or tracks.
Would you say like dividing? So dividing into like, oh, this is like a low commitment level group and then having prizes for that or like a And then.
I thought.
Yeah, exactly. It's like, you know, I feel like the RNA seek prediction challenge was maybe pretty daunting.
To you know some people like myself who aren't experts in RNA seek but you know, having like other smaller challenges where Yeah, maybe the requirements are very basic.
That, you know, that could be an entry way for people to then get further involved. Cause you know, they spend a little bit of time with the first part of the contest and they say, maybe I can do these other parts.
And that could just be one way to get kind of the broader population involved.
Thank you. Yeah. Bjorn, were you going to say something?
Nothing that you for I thought I I mean the idea of like bread crumbs in terms of like, you know grouping people in.
With something I thought that was the main point that building was making and what's easy for some actually for many people RNA seek is going to be the simplest.
Because that's what everybody has the tools for. Who's done bioinformatics? Courses and they don't actually understand and you chose like cell frequencies and other people are scared by that.
So for for some people what you consider easy is difficult for someone else, right? But the idea of drawing people in.
And then, you know, expanding and not. Not asking them to understand everything and making sure like that yes if you only do this one thing It's super cool. That would be awesome.
Thank you. And Mihita, thank you for your comment in the chat. About the Caspian, say PRI competitions.
We'll definitely look into that. Thank you. And also since you are all people that engage in, we're participating, if you know of people from your lab or your university or institute, that, you know, they might find this interesting if you could also share that I think that would be a big help for us so Okay.
And then the last question and most importantly, we wanted to ask what did not work for you that was not captured in the feedback pool.
You know, what could we from the CMI PB team do better? You know this is we just wanna know you know what work what didn't work yeah If you can unmute yourself and Time in that'll be great.
I can I can get the ball rolling here I think so one thing again having participated on both the first and the second is that I think it's less about a person and specifically what someone's doing, but it would be interesting to know about the tasks going forward and what we've learned and what tasks we want to pursue, I think, in the next one.
I saw a question about the difficulty of tasks, but I'm curious, do we plan to Use the same as a still up for consideration which tasks we're going to use going forward for next year and I really like this idea of difficulty and using the knowledge right of these different years in that.
So will that and will that be incorporated, do you think, development of next year's, next year's tasks.
That's a really good point. I didn't think of that. Obviously because difficulty we can now quantify and saying like how did people perform last time, right?
We have a difficulty based on last year. Also another thing that hasn't come up yet but This time around, we're gonna have a lot more participants.
That was another concern. That case based on our relatively small number of So study participants, we're gonna have lots more data.
So the differences in whatever correlations and so are gonna be a little bit more meaningful. So yeah.
I think Jeremy, you're right and we should be pointing out for some of these.
I mean, minimally we can say that, these kinds of tasks performed, were sold pretty well by these models.
Try to improve on them versus these other ones where Not really, right? I think that would be really cool.
What do other thing?
You know, I really love the diversity of the approaches we just heard about, right? It was really energizing to hear, you know, how William was doing it and the thoughts that he had and and how they Minnesota team was doing it.
You know, and there was a large number of. Similar methods taken, so a real stance is, is not.
Funneling everyone down the CM. Root in some way, so maybe not doing that as well, right?
Yeah, I had a suggestion. So my thought is, you know, people when they come, maybe they don't know.
Too much about and maybe really making the participants understand like you know there's other types of diseases that are similar and you know you're providing like cytokine label sequences providing a way for them to really understand to see the biology with their eyes say, wow, this side of can level is helping prevent, let's say, inf influenza or helping when you do get influence, how does it improve your
immunity? Think people if they had maybe this is more related to the zoom but really being able to see that what part of the data you know which signals which cytokines would they how are they working and how are they providing that that boost in immunity or just help with immunity.
I think people maybe it could be whether the content on the website or like a YouTube video or a zoom information maybe people get connect a little bit more to this idea of they really feel like the data is connected to you know improving health something.
And maybe tailoring it to people, are you interested in in health and trying to solve it and maybe for those who are more like data science focused.
You can really try to make sure people understand this is kind of like a Kaggle. You're not going to say I'm putting on a, but really making them understand this is for anybody.
To join.
Yeah. Thanks, working for us for all the inputs. Yeah, I think that yeah, we we have put we have very simplified date like data and the description about each aspect of competition on the website.
But yeah after your comments I feel like there is a more way to go about it. Thanks.
Any other comments, feedback?
So I wonder if we could talk a little bit about, you know, the. The kind of harmonization and the batch effects.
They were in, they were processed differently by different teams, is it? Is that where the secret sauce lies rather than the method afterwards or is it?
Now is everyone starting from the same point or you all? Going in from different starting points. Because the most
It would be really fun to compare different models against. The same harmonized data and see how that does, like from the winning team, for example.
Exactly. Yeah.
Yeah, I remember when honest, in the first challenge, she used, this normalized data really seemed to help quite a lot and then again we see the winner with really highly normalized corrected data.
I think Yeah, we absolutely need to try to see if the different models. How they would perform.
Yeah, I mean, one thing I should say, like that, 2022 data you added, there was strong bad benefit.
It was not that strong in the previous year datasets but this one definitely needed the batch correction.
So that that could be it. You know.
Anything else?
One thing that I'd like to comment on is, so I worked with, Barry and we had a couple of masters, students and data science that, you know, work together.
And I think one huge hurdle that we had to go over was actually explaining. Kind of what genes are and stuff like that and that so I really resonate with what and mentioned about kind of increasing the accessibility of the resources.
And so while a lot of the explanations were relatively simplified, I think even going. Further in order to engage kind of like Data science students.
Hmm.
Broader community I think is very, very helpful. Yeah.
Do you think there's a preference on those kind of resources? Do you think if we delve more into the website that would be?
Okay, or did you do you think? What would be the most helpful?
I think, I mean, I'm speaking for myself here, but I think a combination of videos like multimedia.
Yeah.
And writing on the website is very helpful. I think, you know, that diversity and the way you communicate the basic sciences.
Okay.
Really important to not keep it too dry.
Cool, thank you, that's helpful.
Okay, and thank you for all the chat comments, so I'll definitely, we'll definitely as a team look through everything.
But with that, does anyone have any last questions?
So again, we'll be announcing our public challenge in around August, 2024 and we'll be emailing everyone, here.
I will also be doing some outreach activities through our different outlets here, social media. Conferences.
And if you again know of any people that would be interested, please have them register on the same IP website so we can send them all the information.
So. Awesome. Again, thank you so much for your participation and thanks to the CMIPV team and the consortium.
It was a big effort, so, thank you everyone for all your work.
Thank you.
Thank you, everyone. Thank you for joining.
Thank you.
Thank you very much.
Thank you. Oh This.
Thank you.
I will be uploading this onto discourse so if anyone missed it feel free to send it to them.
Thank you. Bye.