13.06.2024What made you decide to pursue a career in IT research?
I don’t think I ever really liked any other option for my life than being a researcher.
When I was around 12, I decided that I will become a mathematician. I was quite convinced that in order to be successful, I needed to decide on a goal and just go for it. When I was in high school, I was in love with abstract algebra. I found all the objects beautiful and enjoyed figuring out new ways of thinking about them. I loved problem solving as well. I was very much convinced by the ideal of never doing anything that can be used in practice, in particular for causing suffering. I was in the Polish Children’s Fund, and I had the opportunity to do all the fun stuff such as attending courses at the University, attending mathematical camps and participating in the Math Olympiad. After graduating high school, I went on to study math and computer science at the Faculty of Mathematics, Informatics and Mechanics, University of Warsaw. Then I resigned from studying computer science, because I loved mathematics.
But then I realized that being a mathematician wouldn’t allow me to have such a direct impact on the world as I wanted. So, I changed my mind, and I was allowed to switch back to studying computer science.
Generally, I wanted to do research, but the kind of research where I could convince myself that there is a significant chance that someone would use it one day. And since the new major with a focus on machine learning was created at MIMUW for master’s students, I decided to pursue it.
Then Marek Cygan was kind to allow me to work on a project with him. And then Piotr Miłoś gave me an opportunity to do a project with his and Łukasz Kuciński’s team. He then agreed to be my supervisor during PhD. So, I think the biggest plot twist is not me pursuing research, but it being much more computer science than mathematics.
Briefly, what is your current research at IDEAS NCBR about?
Currently I’m working on understanding how to make best use of pretrained large language models in order to have good embeddings from them.
Language models trained with the language modelling objective (so the way that language models are usually pretrained) don’t produce good embeddings. Good embeddings mean that sentences that are similar should be mapped to points that are close and sentences that are different should be mapped to points that are distant. For instance, sentences “Apples are healthy.” and “Eating apples is good for you.” should be encoded into similar vectors, while “Tomorrow it’s going to rain.” should be encoded into a vector that is distant from them. To obtain models having such properties relatively cheaply, it is possible to make use of the general language understanding that pretrained models have about the language and finetune them. My goal is to find how to make this finetuning process efficient in terms of the compute that is needed.
This is needed, for instance, to perform retrieval – which is one of the ways to overcome the limitations of the current chatbots, such as the difficulty of verifying their answers. It’s an extremely exciting research direction, because while not being on the applied side, the results from my work still have a huge potential of being relevant to many people.
Do you think there is a gender disproportion in the number of AI/ML experts?
There definitely is a gender disproportion, I’m the only woman in my group, among more than ten men. When I was browsing professors at different universities in Europe, I noticed that there were barely any women among them.
I think my supervisor and my older colleagues are great and they definitely are role models for me, but I would really like to have at least some female role models. I know they exist, I met some, but I have not had a personal relationship with any of them.
What, in your opinion, could change that situation?
Today it is still quite easy to get into AI/ML research, when comparing it to other areas. The reason is that it’s so new. Allowing someone from an unconventional background into this world can be beneficial, so women could be encouraged to pursue a path into AI/ML research through such a conversion.
That said, I’m not entirely convinced that women have the hardest route into IT research, so only focusing on this group might not be all that fair, for instance, to people coming from smaller towns or less developed parts of the country.
I moreover believe that in an area that is as fast paced as machine learning, encouraging more men to use a significant portion of the parental leave available to them could have a positive impact on the disproportion between the number of men and women pursuing it. At the moment, not working in the area for a year could mean falling completely out of the mainstream of research.
How can women contribute to the development of this area of science?
I don’t personally think that women and men are different in any way that could cause a benefit for basic research in ML if more women joined the ranks. So, they can contribute in the exact same way as men do.
People born as males and females have different bodies. For many years, research in medicine has been conducted only on males and then generalized to females. Despite the efforts to change that, this gap is still present in modern research. This is a significant issue, since symptoms of some illnesses might be different in females than in males, which might cause women to be underdiagnosed for many conditions. Moreover, the efficiency or side effects of medications might differ as well, causing women to be undertreated. Therefore, in areas immediately adjacent to medicine, it seems like more women could influence the research direction so that this inequality would be minimized.
Are there any related cultural factors that could encourage women to pursue their careers in AI/ML or discourage them?
I’m not really sure. I definitely feel that some older researchers seem to keep a greater distance from me than from my male colleagues. This is discouraging, since it makes the networking more difficult.
Many people have sexist beliefs. I was told by teachers as well as by family members that doing math/computer science is not for women and I am definitely not the only woman who experienced this.
If you were to point out one character trait that describes a good scientist, what would it be?
Confidence – in order to pursue an idea a scientist should not stop when their solution doesn’t seem to be working initially. And then to present it to others and protect it from their critique.
Alicja Ziarko is a PhD student at the Doctoral School of Exact and Natural Sciences of the University of Warsaw. Her research focuses on making use of internal representations from pretrained language models in order to facilitate reasoning. See newest article co-authored by Alicja here.
