Or the very first. Cmi PV informational session. This is part as, Primote will explain in much more detail.
Of our computational modeling of immunology. Project where we're trying to predict. The outcome of, purchase, booster vaccination, which is part of the T-dep.
This is a grant that was started. Just before the pandemic started, so like many such grants we are behind on our original timeline of what we wanted to do.
But, the basic idea is that we are trying to Provide a forum. Where people can compare different approaches.
To computational predict vaccine outcomes. In the setting of. Protest booster and then get an idea of what actually modeling approaches work.
Better than others and ideally learn from each other. And this is our first public attempt to do so we've done a version in the past where we did this all in-house.
And this is very much a learning experience for us. So we really appreciate feedback we also appreciate patience If things don't make sense.
And, Things don't work. Just contact us. this, we are all in this together to some extent.
And with that, Primotech should take over, from what's being a post, in the lab for pretty much since the start of this project.
So for over 3 years and has really been a fundamental in establishing all the efforts to capture data and, to set up the mechanisms how we can.
Terry, out, these evaluations. So with that, Pram, please take it away.
Yeah. Thank you, Beyond, for, the overall introduction. And it's nice to see such a significant number of participants joining us for the first informational session.
There are many participants who have requested for the recordings because of like they are working from in different times zones and some couldn't make.
Today, so after the meeting, we will upload all the recordings on the, our discuss forum.
So, so going with the, presentation, next slide, please.
So the prediction challenge is. Pretty can challenge is the contest or a community event in which individuals or group make predictions about scientific specific outcomes often based on their knowledge and expertise and doing like lot of different analysis.
And you can see there are a couple of examples. I've shown here the first one is more general, the Netflix price which which, like which completed in 2,000 7 where 1 million dollar contest was organized to see how people watch, like that time long videos and it has forever changed the way like now we are watching binge watching.
And one of the very prominent competition was started in 1994 and like I think last year it was sixteenth Welcome, community version where, like participants from all range of, industries and academics were asked to predict the protein structure.
And one of the products I want to emphasize here which is the Deepmind which has predicted the structure of all almost every protein known to the science.
There are a couple other examples like which are less famous but like they are like but our various agencies are now going into the route of prediction contest because for one group or set of groups, sometimes some problems are not so solvable and they want a community efforts.
To come together and solve these problems. There are so many different problems, competitions are continuously growing.
In science, STEM field. And these are the 2 famous or pronounced examples of those platforms like Kaggle from Google and the Dream Challenges, which is more for the science.
And, and so what are the benefits of running these prediction challenges? So. So after the challenge, the evaluations are quantitative and unbiased in nature.
These challenges drives participants to strive for accuracy and outperform the other peers they encourage wide range of participation.
If I take the example of our contest like we We want like, people from like different backgrounds should come and, solve or try to, solve the task.
And yeah, this encourages people who are from different backgrounds as I mentioned and it contributes deeper understanding of our topic in hand.
Next slide. So what we are trying to solve with this system vaccinology challenge, so if you see system axiology data it is varying in like it has the steady design is varying for any any any people you take and data is generally multi-dimensional and heterogeneous in nature.
There are so many different parameters for instance if you take the BALCOS in LUCLE RNA like the number of parameters in thousands but if you see the steady participants it's in the range of like most of the studies until before COVID it was like 20 or more like in some cases 100.
Now the number is growing but still Most literature shows like there are few steady participants and there are lack of methods to test the generalizability and the prediction performance of all these models which are predicted in a sense that if one model is developed on one cohort or data set or a particular vaccine it is a little hard to reuse that model for other dataset or other vaccine.
And also, SIS and Vaxenology data has variability between individuals, which is key to ensure that model performs best.
Also, like, generally like system vaccination studies or also infection in in last sense they study pre and post study designs.
So temporal dynamics is very important or key to go in detail about understanding the biology behind it.
So with these MAPPB, we are trying to establish community platform by running experiment and developing data which will then leverage to develop and test condition models of immunity in vaccination setting and particularly we want to better understand vaccine induced immunity to buddler puttosis.
Okay, so in today's session, there are different topics we wanna cover. And like I'll start with the introduction about the project.
Then how we wanna. I, we have established MAPP challenge and like how the all the invited participants you are going to access the the invited participants you are going to access the resources then I'll be briefly talking about the all the invited participants, you are going to access the resources.
Then I'll be briefly talking about the models which we develop in first challenge and one simple model we develop for the second challenge for for like for demo purpose and then I'll also talk about the other resources which we are providing from the via a same.
We website and of course large large amount of time will be doing question and answer with all of you. So let's do a deep dive with the.
So, but. Buttas is our opening cup is the disease. Caused by sorry but digital apprentices which is back to them in upper respiratory system so it has acute symptoms like low fever, Renino is mild curve apnea in infants and the chronic symptoms if disease persist for more than 3 weeks.
Oh yeah, for example, like patients can develop cuffing fields often ends with the oof sound and and there are many complications and 2 reduce or to control this disease.
Historically, I mean, since 1,940, vaccination is available. Which which I'll be talking in detail in next slide.
So if you see the number of cases in US, this is the plot which I've taken from the national notifiable disease surveillance system, which is the entity of CDC.
So you see the in the beginning of nineteenth century, the case load was in 200,000 and then the first vaccine, vaccine which is inactivated whole baptism vaccine or WP vaccine.
It dramatically decreases the caseload and by 19 eightys almost like there was a There is no, very, very low, case load and, then, there was observations of reactive generosity of, this WB vaccine.
And then there was a discussion about Can there be a new vaccine? Should be develop, to control this reactor, So, so in 1995, US implemented a similar version of this vaccine.
And you can see that then, stay, then after introducing this vaccine in, in few years, there was little jump in number of cases.
Which we saw, and, particularly in, California, it was developed, it was declared as a pandemic into 2,011 and also many other countries have started to see a little jump.
I run this time after introduction of AP vaccine. So this is a kind of interesting, study we wanna do a little, deep dive.
It's, it's because our motivation of entire project is to study the veining immunity coming from the new or acylular purchase of vaccination.
Next slide. Oh, it's now I'll invite Lisa to briefly talk about the recruitment strategy and, experimental, data sets.
Yes, thank you, Pramod. So my name is Lisa Vincent. I'm a postdoc in a peers lab and I'm taking care of the donor.
The data generation of the donors. So for this project we are in rolling new donors every year.
And we aim to enroll 16 donors per group per year. So we are we are enrolling in adults and these adults got either the new AP vaccine or the old W.
Pack vaccine during their infancy. And when these adults are enrolled, they got 10 different plot draws of which 3 of them are before the TDA booster vaccination and 7 of them are after the boost of vaccination.
And we are using these blood draws for the data generation. So we are splitting them into PBMCs for the moon cell characterization and for plasma for other measurements.
Next slide, please. So we are using the PBMCs to measure different cell frequencies by flow setometry.
So we are for instance measuring 37 distinct cell populations and you can think about different Tesla populations for instance.
And then we are using the plasma to measure antigens specific antibody titers. We are using lumenex for this.
And we are measuring the IGG isotypes against the different vaccine antigens. So antigens of protesters, tetanus and diffteria.
Hey, we're also using the plasma to measure different cytokine concentrations of the blood.
Think about for instance in from gamma And lastly, we are using the PVMCs to isolate RNA and then we are performing book RNA sequencing to get an idea of the transcript home of these cells.
And meanwhile, we are still working on adding more analysis for these for this prediction project. We are for instance working on adding a more specific TCL and B cell, experiments.
So you might see those in the future as well. And with that I would like to give the word back to Prima.
Thanks, Lisa. So after running this immun, human characteristic assays, we are generating a lot of data.
So now this is essential to organize this data so that contest participants can effectively download and access this data for the model development.
So apart from the CMIB project, we have a data coming from external funded projects and a lot of metadata is coming from the literature.
So we have assembled this data in our central database and from there. This database is available as API or direct file download for for downstream users.
So we usually like the first step is to do the user trustings with our internal labs and it was so far it's working nice and now this is the first instance we are opening our database to public users.
So now this is a little technical. I won't go in much of details, but broadly, our, our database schema looks like this.
So it has categorized into 3 broad category of table. The first one is subject and specific in specific information.
Second category is experimental data sets and third one is metadata. So if I give an example of subject tables, so all the subjects are in the study, I have unique subject ID and associated information about infancy vaccination, biological sex, ethnicity, and so on is It's, constituted as, a table.
Then if I want to give example of experimental data sets, so for example, the gene expression data, coming from a PBMCs.
Has like each row in this table has unique specimen ID then, version on similar ID and, information about Rock on and the TPM.
Then we have associated metadata.
Next slide please. So after this, will be briefly talking about, the CMIB challenge, like the way we, open this data, in more accessible format.
Next slide. So, so see there are, layers of, challenges we have plan.
This is more of learning process because system technology challenge has not been previously reported and this is our first instance.
So we started with the first challenge which was more internal dry run. Where SIMa PV consortium 3 labs participated and each of these labs have given training data set.
And the test data set. And this city has been, concluded and, we have, shown the results and we drafted this results in by archive.
Like manuscript which is accessible. Now we are at the second challenge where we are opening this challenge to invited, invited contestants like you all where, We got it.
We are. We are, clubbing, training and test data sets from the first challenge, and providing it as, training data set for this, the current challenge and there is new cohort which will be provided as the test data set.
I'll be talking about the exact data sets in in following slides. Then we have our third and fourth challenge, which are more public.
But with the current challenge, we wanna understand how, how this challenge works. Next slide.
So this is a brief outline about these. Might be challenge. So the challenge has been announced and materials has been populated on the website and associated webpages like APIs.
So, data is accessible there, then, we'll have to, we all to, build your prediction models and, and, at the last step, we would love to have your predicted, like submissions.
To be submitted by our website and then we will evaluate those results those submissions in inform of the results.
And next slide. So the very first thing about the challenge is the questions or the prediction task. So since I'll mention this is a kind of a new era.
So what we have, done. So we identified where our accent responses, from our one of the previous publications and these vaccine responses were formulated as petition task.
So these are 2 kinds of responses were formulated as petition task. So these are 2 kinds of responses, the general and AP WP specific practice responses.
So if I talk about general vaccine responses like Plasma IGG levels of against participant vaccinations.
We we saw there is increase. In in levels at day 14 similarly we saw frequency of monocytes increase post vast post booster vaccination at day one.
Similarly, we saw, CCTV, in expression is elevated at day 3.
Post vaccine, post booster vaccination. Now we we took this information and went ahead. Next slide, please.
So we formulated these, like these findings as a, as a prevention task. So there are 3 categories, as I mentioned, and will title task, which, which is basically one task divided into 2.
Similarly, we have cell frequency stars and gene expression task. So briefly, the the first category of a sub task where we ask, we ask participants to rank, individuals.
By IGG antibody titles post with that we detect at day 14 and second task of task where we ask to submit full change values and now Now, next.
So here, the ultimate goal is to model as many as task possible, but, but we, we, it's, it's, it's also good if we, if you are expert in one of the data sets or the assays and you, you submit one on one or very few tasks.
So this is the example of rankings, which, which I'm showing here. So, So there are let's say, like 5 subjects and you predicted some values for one of these tasks and so we want you to convert these predicted values into the rankings like a higher number should rank one and the lowest number should should rank 5.
Next slide. So here I'm, I'm showing the, like actual, submission file.
So the first or upper, panel shows the complete version of submission file where it has information like column A to D.
It is the. Information about the subjects which are there in dataset and other remaining 6 columns are the task and we we want to you to fill these cells with the ranking informations.
And then the how the partial submission should look like so this is example so wherever you attempt a task so that column should be field and and if you don't attend the column the task the column should be empty.
Next slide, please. So. Now, let's talk more about the data.
So, so, Lisa and Lisa explain like those, data is generated by experiments and now we have these large data sets coming from experiments.
So, so the first thing which we do is to assign a unique subject and specimen IDs.
To all the study participants and samples that we have. Then the next step is to go inside each assay and identify features.
And then label them with the unique like unique IDs. For instance, like for all gene IDs, we assign, on symbol genities and for all the proteins we assign unit proteins.
And for, antibodies, anti, we have assigned ontology terms which are there in specific asset data sets.
Then we also ensured file formats and data formatting is consistent across all these. All these asses and this harmonized dataset is now like deposit into the database.
Now, there is, another thing to note, which is like the second challenge.
It involves the data set from the first challenge. And like the ring and test data set so there are basically 2 independent data sets that needs to be merged so So, so CMIB team has worked on this issue a little bit and and providing you process data or compatible mattresses where where like contestants can just take those compatible matrices and build on the modeling.
Bye. But, Bye. But if you are expert and you don't agree with our data processing or compatible market generation pipeline.
We welcome you to develop your own pipeline and do the data processing. Now the like processing, like data has divided into like after data going into the database.
We have divided into 2 categories. As I mentioned, training and test data set. So for trading data set, we make available baseline responses before going, giving the vaccination and all the longitudinal responses which are after giving the vaccine.
And for test data we will provide you a baseline. Immune responses but we will help back vaccine response data data set.
It's because we ask you to predict the, this is responsible. Responses for particular task and then these vaccine responses which are hidden will be used to evaluate you a submission and and and and and and like best models will be evaluated all this information is already available on our website and feel free to access it and if you find any issues, let's note.
Next slide. Now. Now, so data is their task variable is there even the challenge information is there on a website.
Now to assist, modeling, which will be done by you. We have providing 2 more resources.
Next slide. The first resources is the, the manuscript and all the, code and our learnings from the first challenge.
Where like 32 models were develop My, CMIBB consortium.
The first team, which, which took the information about the models which are already published and we reapply published and we reapply those models for the models which are already published and we reapply those models for the CMIB data. So those results are there.
Then second team and third team which are Farahab and Steve Ginsen's lab which so they are expert in multomix data integration.
So they're developed, Jive and NCI based models. So, so the code and all the stepwise interpretation of their analysis and results is also we are providing along with this manuscript and on on the our GitHub repository.
And next slide. So just briefly talk about the best model which we. Which we identified during the first challenge.
So as I mentioned, like, this, Vaxxinology data is a lot complex and, to come to do modeling, the data needs to be processed and and So, so that's why the first step which this and say approach has done is to do the data.
Imputation. So the first step, which was like doing data harmonization and then they saw there are a lot of missing values which they I'm building using this mice algorithm.
And then in the step 2, they build 3, different approaches because the first, contest was more open kind of so we asked like whatever models you develop you just submitted by each team so this team submitted 3 3 models.
The first one is baseline approach where they in they took clinical features like age infancy vaccination and biological sex and all.
14 tasks which were given for first challenge they integrated this data and You can you can see here like, they build a last integration based, prediction model.
Then the second method was the mode, Melomics integration. So they took the data, they built, Multiomics fact latent factors using MCI method.
Then these global scores were then used, to predict each task using the same last integration. The third approach was the combination of the first like and say a basic and baseline approach, which they called as MCF plus approach and, There were they combine everything and and and did the prediction Now, if this is our next slide, so.
No, I'll briefly talk about the results from the third team or or Steve Kleinstein's lab where you can see that these models are performed better than all other prediction models.
And you can read about the details about like how these models perform outperform other models in our manuscript but also if you If it was confusing to learn about this, feel free to ask.
Awesome. Next slide. The second model, not exactly the model, but we want to provide a resource where we use the baseline response.
Response of each task variable to predict the vaccine responses. So the goal of this pipeline, which is provided as the our notebook, is to illustrate how to access data, how to do like here the simple modeling and how to prepare a submission file.
So we use our training data from the second challenge, 2021 plus and we use our training data from the second challenge, 2020 and 2021 plus and we calculated.
Experiment, a correlation like for CCL 3 task. So we took CCL 3 day 0 expression values to predict and and submitted as as like predicted values and then calculated.
Like a response. A calculated experiment, correlation with the response data, which we already have because this was the training data set.
And you can see that it's not like perfect model, but you can start to see that there are nice correlations are coming out.
But of course it would be interesting to see how this coalition worked with the test data set. Next slide, please.
Now, let's talk more about resources that we are providing, with the, with the SAMIBB website.
The first resource. Is of course our website where we have summarized every piece of information that we have generated until now from data from our understanding and and all the question is there where which we discussed in the past.
So the first thing which you'll see a big banner on the top of the website, where we are providing challenge data set, training and test data set.
You can just jump in there by clicking here. The prediction task list is also available there and the manuscript also like link there.
Then if you want to learn more about CMIBB project, then purchases, disease vaccination.
So all this information is available. I'd learn about the same IPB page. Then we also have 2 dedicated pages for understanding data, our database schema, etc.
And and and using the access data you can just jump in and access like not just the current data but all pass versions of databases through this page.
Now more interestingly we have developed this prediction challenge page where you can access each and every piece of, information about the challenge.
Then the last one, like after, spending some time on this, if you come across a lot of issues.
So we have play that is called as the solution center. Next slide. So like here in the past, we have asked lot of questions and these communications are like we have preserved and like these are right now read only if you don't have account and for instance like this is the question somebody asked and and then we internally discuss and come across this answer.
So you can read without an account, but if you wanna contribute and participate in asking question. Then you need to get an account so how do you create the account next slide please So to create an account, like on the talk, you see sign in CMI PV option.
Just hit on that then it will ask you to sign up. And when you click on sign up it will ask you to fill some basic information or also you can just click on a single sign on option using the Google and then the account will be created.
Next slide. So similarly, like as the next step, how to submit a prediction. So you'll see after creating account.
A submit prediction option will be available. Just click on that, then it will redirect you to the page which is dedicated for a submission with like brief steps.
So you choose your file and, do a submission from here. Next slide. So now, I'll, I'll, I'll request Mari to briefly talk about the challenge timeline.
Thanks for mine. My name is Maria. I'm a project manager for the CMI.
And I'll be talking about, the timeline for this challenge. So today we have our first informational meeting here on October sixth. We have 2 more after that.
So our second informational meeting will be on December first. And then, 24 will have our third informational meeting on January twelfth.
And then right after that, we'll have everyone submit their models and the final submission today would be January sixteenth.
And after that in February on February second, we'll do an announcement of the winners and we'll also release a longitudinal test data.
And again, if you're unable to make any of these informational meetings, no worries will be recording all these meetings and posting the slides on discourse as the one mentioned.
So if you can't make it, but if you want more information, let me get totally go there and also, submit your submissions without going to any of these.
And then that one. Yeah, thanks party for briefly talking about the timeline. Now, this is the, CMIPB team we have.
So, PETerslab, like, there are so many different people, like there are so many different people, they have expertise from like learning experiments, they have expertise from like learning experiments, then working on databases, from like learning experiments, then, have expertise from like learning experiments, then, working on databases, working on, web servers, etc. So we are generating data and populating this data into, the database.
Then we, we have, Baris Lab who is developing, teaching materials for us so that the information about SIMPB can be taught into into the inner city settings.
Then we have expertise from the Steve kind of Cleansen's lab and Faraday's lab who are expert in big data.
So they are assisting us with the like what is the best way to like integrate or store data into the database and do the basic pre processing to ease the model development for the participants.
And we also have, like, we want to thank, SET, and, LEVE and KURTLAV who have developed in our institute how to establish experiments for, launchunal response studies, then, we wanna, thanks clinical code and bioinformatics score from our institute who are helping us with the I recruitment and data generation.
So this is the team. this is the, like we are doing, information, session on zoom.
So I thought to just put, faces and introduce you to all of these so yeah And now I open the platform for questions and I think a lot of people from CMIB team are there.
So, yeah, feel free to just jump in. To ask or answer the queries.
Thank you.
Awesome. If anyone has any questions, if you could either raise your hand or unmute yourself.
Or feel free to use the chat as well.
So I have a question to the participants. Has anybody here ever participated? In the challenge like this. I mean, I can personally admit that I have not.
I've always followed with, fascination specifically the cast. Contests which are always stop us that was my the guiding light, but I've never done it myself because it was never in my area.
So I'm wondering if anybody else has any personal experience.
In ever submitting anything.
I participated on CASP a long time ago.
It doesn't show.
Casp I hear from someone. I sorry I didn't catch who that was
Yeah, that's me, I participated on CASP for Production. That was a long time.
I would be very curious how How does this compare? Did they have similar sessions like this? Did they?
No, they didn't have any session. Just release the targets and cast had a very short like they released the target you had a day to submit the model.
Yeah.
In the model is the solution. It's not the that they're going to test. They just, they give you the sequence you give them the structure.
They don't care how you do it. Actually, if you if you did it by hand, you can, but they don't care how you do it.
That's why they give you a very short timeframe. Guarantee that people aren't that there's not a lot of human input.
Right, and then I mean, so thanks Rafael. Obviously we are aiming for the same.
The approach you can whatever you choose to do your predictions you also just have to give the answers. I think the difference with CASP also why they didn't need informational session or so.
That has evolved. Over decades and I think like Gradually grew into what it became versus we are trying to obviously jump start here.
So the point of a session like this is for us to figure out. Yeah, what kind of information is needed?
Alright, so I'll jump in with the first question. So, my name is Tally now.
I'm super glad to participate in this. Just a quick clarification at the beginning here.
So we can submit up, first of all, I want to start by saying that the website is fantastic and I am super thrilled with the team.
This is clearly hours of effort to do all that so thank you I found the data I found the challenges a lot of the background is on there So a few questions that I have.
So, You can submit up to 5 times before the due date. Do you get evaluated on those submissions?
Like you learn which ones are right, which ones are wrong, or you get a spearmints back.
I think that's a nice question. So what we have learned from the first challenge is is that like modelers they start like modelers they start like they work on the first challenge is that like modelers, they, start, like they work on the first challenge is that like, modelers, they, start, like, they work on the, their model, they generate, a sufficient files and after submitting the file they realize that, oh no.
like this type is missing and if I do that maybe the predictions will be improved. So they resubmit it.
So that's why the 5 number is there. Thetically or we want to like we have make it clear that we will evaluate the last submitted file.
So it could be like number 4, we will evaluate the last submitted file. So it could be like number fifth or number 4 or just one.
So that will be evaluated . For sure. Yeah
Okay, so only once is it evaluated. That's the very whatever is there at the deadline basically.
Yeah.
Yeah. Okay.
And, we haven't talked about this, but if somebody says I have 2 approaches.
I think we should allow for that. If they're saying, I mean, I want to. Whatever, I have my deep learning approach and I have my See, heuristic.
Whatever approach. so do we tie it to an email address? Is that the?
Yeah. So I think it would be nice like if you have 2 or multiple approaches, just create a separate account for each approach like separate email account and and manage that kind of a methodology submission.
Bye-bye that, like, email.
Yeah, and I mean if this comes up and people don't have a hundred email addresses so that we can work with them.
Okay.
We can, I'm sure we can work around it. I mean this is a good example of something that we can obviously do manually.
Okay.
Yeah, yeah.
Awesome. Does anyone have any other questions? I do.
Hi, this is a Joe. Yeah, I have a bunch of questions.
So go ahead, start.
Yeah, yeah, and this is a team of, James Kobe in our University of Alabama and Ringo Huang and a Joho and Fred Hodge so we have exam this data set already and I have a bunch of questions so First one, I just want to know, you know, because you mentioned, you know, like this data has been like processed but is there like the data or
training data set you provide you already split to like like the 2,020 and the 2,021.
But I want you got a clarification you know I have you being like the normalized data across, you know, this 2 years, you know, remove the batch factor.
Yeah, so, sorry, I didn't explain any detail. But so that's the idea.
Like we have 4 assays and these are club and like the agenda, this data sets has been generated in 2 different years.
So, so like there are different issues. The very first issue was like overlapping features. So there are certain experiments like, O link.
So the first year we had like very larger panel and then rolling stopped working on that panel so we have much more targeted panel.
So like you have this not issue but like feature overlapping identification problem. So that's the first thing and second thing of course the data distribution was completely different.
For certain assays, so for instance, RNICK, the 2,020 and 2,021.
So average expression values of this data, 2 data sets were completely different. So, so, so these 2 like kind of, data sets have, issues.
So that's why we needed to do a pre processing or data processing and batch correction. So these things have been integrated into the pipeline which, which we are providing.
And the code and data both we are providing. Yeah. I think beyond, you, you're muted.
You're UN.
And you do.
Yes, talking on mute, my favorite thing, sorry. sorry, I think the question was, Pramod.
In so far with the current data set where we have the baseline values was that also batch correct it in the spirit.
No. No, we kept it separate. At the moment.
So what exactly was Normalized or harmonized or any kind of anything whatsoever.
Yeah, so, access, like, the, the expression assays from 2 years.
We took them and then we applied normalization on them. So, so from the dating date that they sell.
Yeah, if I understand your question.
Okay.
So that was PBMC gene expression, all those TSV files, basically. But TBMC cell frequency is wrong.
Yeah, no, like, we did it independently on each assay for, then the, side of data is independently processed and normalized.
Similarly, oiling data and, andability data is separately processed and normalized.
I think.
How is the data normalized? How is the antibody data normalized?
For integrated data, we saw initially, like the data distributions were completely off.
So we took the baseline, median of, of each feature and we take this not a, factor and divided it to like entire data set.
So we did it for like year one data set independently and here to data set independently.
So that's the like MFI. In the antibody tighter data MFI is the raw and MFI normalizes your attempts to normalize between datasets.
Bye. Okay.
So we can grab both if we want it. Okay.
Yeah, and that's our spirit. Obviously, normalization can be done any number of ways. This has been our as our recommendation for normalization and that's what we also using internally for our own models but we are not at all sure if this is the best way of doing it.
And I mean, this was already, a, that you have a description in detail how it's done, right?
Maybe let's make sure that that's already accessible because we can't discuss it in detail here on this call but this this is a challenging question and what Primat was already saying like I mean we knew this was to some extent the problem But what has been an additional problem is that year over here the companies change things.
So if a company no longer offers. 3 agents that you used in the last year or in the case of Oling just completely drops the whole a whole set of essays.
We were in travel, right? And this is, actually happening more often than not. So, yeah, and we would also love to learn from others.
I think the best. The most, how do you say? The thing that makes me feel the best about is that the final predictions are meant to be ranked.
That's our primary output, rank values. All the normalizations do not mess with rankings.
Alright.
So, the highest to lowest order is always, retained. Sorry, and I see here a question from Aleksandar, Barcelona, in the, I hope I didn't screw that up in the chat about can you elaborate on the publication at the end?
Is there policy on the freedom of participants to publish the results. Once the data is made public, there will be complete freedom.
So what people we will definitely have a joint publication and we would like to invite everybody who participates in this contest to be a co-author on it.
But then there might be this might be short, right? In, in there you might just say, Hey, the method so and so, so, so, so, so, so, so, so, so.
Maybe the top 3 people get a paragraph, but you might wanna have pages. In which you wanted to ascribe it and there's gonna be no limitation at all whatsoever for you to do that.
We are kind of anticipating obviously this will be it makes more sense to do this in the same issue or after the general evaluation is out I mean you might have a harder time.
To publish by yourself. Before the contest is explained. I hope that makes sense.
Alright, thank you. I have another few questions, you know, could you just explain, you know, like the 2 goes.
Because you say you want like the prediction from the like the mono side and the same question for like the IG but in the data set actually you have like a multiple mono sites you know like the classical non-classical and something you know quite a similar and the same question like the full I you'd hide you also have some like sub isotype IG.
So could you just, like elaborate, you know, what exactly you want to us to predict?
So if I may be, I hope the question is really exact. There's I believe one data set and I believe it's classical model size that we're looking for.
Or was the total monoide frequency? But they, we must make sure that they are really exact.
They should always be one. We have a lot r, data in general. Then what we're asking questions for and we are hoping people who have a deeper, I mean, logical understanding might be able to utilize that.
We had in the last round. A much larger fraction of a number of questions. With like the different IGG subtypes or so and our internal experience was that that confused everyone even more.
So we kind of selected 3 questions that are representative of the overall data set and that are the most robust effects that we had observed.
In every year. So, yeah, are these the best questions? You'll, we'll find out.
But, yeah, but they should definitely be, yeah, very, very. Key and explicit what they are if they're not, we, must, Make sure that that's correct.
So just let's dive into that because that is clearly very important. So it looks like it's IGG for the antibodies which is not IG one not ITG 2 not IT 3 or 4, but just the ITG label and for the monocyte that's just monocytes, not classical monocytes, not any of the other ones, just Pure vanilla.
Yes.
I believe so and I believe there should be a column that says model side frequency and there should be a column that says or a row.
That says a total, whatever. Was it? Yeah, and not the sub types.
Okay.
Yeah.
And actually, I love the sub types. But those data sets are much more noisy. So that's why we take the total IGT.
There might actually be more interesting biology in the subtypes, but. Yeah.
And 2 more on that, 2 more questions. Whether we're trying to predict the for antibodies and the fire and the phi normalized.
It shouldn't matter in terms of the ranking because the normalization is done on the data set as a whole for each year.
Let's see.
Can you put it one more time?
So we're looking at antibody data. One of the anti files you give us MFI and you give us and the file normalized.
Okay.
And we are trying to predict, I don't even know what this matters, but we're trying to predict, let's say, MFI.
You do a perfect job. We'll get the same ranking as if we were trying to predict MFI normalized.
Yes. Yes.
Okay, great. So that's like and for the . For that for the gene expression, are we?
Can you explain what TPM versus Rock counts is?
So, there are different measures of, I think majoring, expression values.
TPM is transcript per 1 million. So if you sum up. Well, the transcript so, so it goes to 1 million and draw count is actual count.
Yeah.
So should they be proportional to one another or? Is it not linear?
And I think AZ is not linear. But,
No, no, no, that's a linear correlation. There should be linear correlation between the 2.
The counts give you an idea. You know, you might have different numbers of counts per sample.
And do you have like better resolution in one than the other? But the TPMs are going to be comparable across samples.
Okay, and so again for that case are we should we focus on TPM or should we focus on
TPM, TPM. They, the counts as a number of tools prefer to work with counts directly and again like if the better idea of like certainty and what the.
You know, are you looking at one versus 2 counts or? Yeah, 10 versus 20.
I think I interrupted you. Is your list complete, Joe?
I have a follow up question about your frequency, cell frequency data. So I've noticed, you know, like the in constant, on inc.
It's between like the train data set and the. Like, like the testing data set about the sale population name.
Like I cross check them, you know, some like the differences. Probably very slight, but I think is that any chance you can just synchronize all sales frequency name across the data set in case any kind of the confusion you know because some setting you know like the labor like the CDC CD.
19 but another data set is a CD 3 positive and a CD 19 positive. So we needed to Get a crystal clear answer, you know, if they are exactly matched.
So I just want to say if any chance we can have like the exactly same. Sale population name for the different data set.
Yeah, yeah, I believe like all, the cell type names, like, or like feature names across different data sets have been standardized.
Yes.
I think the answer is they should be. If you found something that's not, we're gonna recheck and if we don't find it that's not we're gonna recheck and if we don't find it it would be helpful.
To But for us, if you can point us to where I would not be surprised, this is also been the struggle.
Okay, okay, cool. Okay, so since on this, so I just want to point another disparity what I found in the Only Sequent Data and the Self Equence and Data.
So I look around, you know, that that figure in your website, you know, they just a divide or sample availability from different like the 4 essays.
And I also check on the data set you provide. I just like the It's all the space man ID and the prejacking the user, you know, in the, meta data set and trying to like summarize, summarize the on the patient ID level, then it end up like I found 74 patients from their PBM say sale frequency, but you say you have 73.
And I found like the 93 patients from the expression level, you know, but you got like a 92.
But quite weirdly, you know, I got the exactly number from the plasma, cytokines and antibody tighter, you know, 74 and 111.
So that 2 parties exactly match but only one subject is off from the cell sequencing and the expression data.
So I need to, you know, Double check, of, guys. I just don't want to make any mistake.
Okay. Yeah. I think that it would be nice if you like, asked this question briefly on discuss forum and so that we can follow in from there.
Okay, okay, cool. And I mean, last question, sorry about that. You know, how did you calculate the sale frequency?
What is the denominate about that percentage?
I think Lisa should respond.
Yeah, so, we first did gating on CD. 45 positive cells and we gated out like that cells, debris, etc.
And then we used those CD. 45 positive cells as a baseline and then when we gate for instance D cells, we, the number is like as a fraction of those CD.
45 positive cells.
So everything is based on C. 45 positive.
Yes, and that's a you moon cell marker in general.
Okay, cool. Thank you so much. That's all a question. Thank you.
Yes.
Thank you.
Thank you for your question. Super, super helpful. Very, very, very happy. And if very embarrassing that we still have, I mean, we work so much on standardizing these names if there's still some in there.
I mean it happens.
Yeah, and they can be confusing because many subsets have overlapping markets, etc. So Thanks.
Also, I think there's one more question than chat. It says by normalization, do you mean you scaled all the values to a certain range?
Say for example between 0 and one.
I think, this is a bigger question that, like, click.
Doing it into the rate so as I said like the average values between 2 data sets are completely off.
There are some outliers in, for example, antibody titled data set. So, we want to read those outliers effectively.
So that was the thing and then, like. The digital distribution is completely off so that we treated with the baseline meeting so there are different things we perform for the normalization and even after doing the normalization for like individual data sets, we saw like there are a certain amount of variation, by variance explained by datasets when we run the PVC analysis so so one step ahead we went
and did the batch QC. Our, batch, effect correction. I think this is the question of walking as well.
Yeah.
So I think what we should be doing is the the questions that came up here in the call. We should for each of them create like a forum entry in the Discourse Forum.
Even if we have it somewhere else, just to also link out. To where that is and I would strongly encourage also because as many people on the call who didn't Get a chance to ask a question.
Please post there. We are trying to do essentially like a self writing manual, right? So that everybody has an issue.
You can have the answer there even if it seems trivial. It's going to be helpful for someone else who can go in and to a control F and see has this question been asked before.
And many of the times it's not trivial, right? If you have any problems, please, please post.
Thank you.
Awesome. In the interest of time, yeah, so if you have any questions, if you could post another scores, that'd be awesome.
And again, we look forward to seeing you in our second informational session on December. Yeah, so thank you everyone for your time and good to see you again soon.
Thank you. Thank you.