09.09.2023What are you currently researching at IDEAS NCBR?
My main area of research is the development of efficient deep learning models through introducing plasticity in architecture and resource reuse.
Current deep learning models typically consists of millions, or even billions of parameters. At the same time research into sparse network models shows that these numbers can be brought down, up to even 90%, by simple compression approaches. As overparameterization is considered beneficial during the training of deep neural networks, the challenging question is whether the same rate of compression can be already induced before (or during) the training. Such approaches lead to efficiency not only during the inference, but also during the optimization.
My research focuses on investigating the theoretical and empirical limits of such sparsification, as well as the practical benefits and opportunities of introducing them in a structural manner, so that they can be used by the hardware. In my work, I often take inspiration from other disciplines, such as physics or neuroscience.
What encouraged you to pursue a research career in AI?
I have always known I wanted to do something science-oriented. Both my parents are scientists. My mother, who is a professor of physics, is my role model – as a scientist, as a mother, and just as a very good person – kind of an everyday local hero, who is also a researcher. In elementary school, when asked who I wanted to be in the future, I would say I wanted to be like Leonardo da Vinci. He was a researcher, but also an artist and I was drawn to his creativity and the idea of a polymath.
When I went to high school, I was in a math-physics-computer-science oriented program. I always liked math, but I did not like physics at school, and was very ambivalent when it came to computer science. When I was finishing high school I thought that I want to be either an architect, because in my mind it was connected to the creative and mathematical parts of me, or an astrophysicist, because I wanted to study black matter and black holes.
In the end I entered computer science studies, as I thought that a broad knowledge of computer graphics would be an advantage when pursuing an architectural degree. The program had lots of math classes such as logic, algebra, etc., and I remembered how much I liked math and that problem solving also requires a great deal of creativity and open mildness. I came to treasure beautiful proofs or programming solutions to the same degree as I treasured a piece of art.
However, I still didn’t love programming. Not until the end of the freshman semester, when for one of the final projects in programming, I decided to implement a mancala game (a type of an ancient board game from Africa). For this task I needed to design the strategy for the computer player. That is how I started to read about AI. I thought that designing powerful algorithms and pursuing AI resembles science-fiction, but also exciting and beautiful. It also finally got me really interested in other areas of computer science beyond computer graphics, so I decided to pursue a degree in Theoretical Computer Science (more math and algorithms than in normal programs), and for my bachelor thesis I started to work with my current supervisor, Professor Jacek Tabor, who is an expert in ML.
Do you think there is a gender disproportion in the number of AI/ML experts?
Yes. Though from mathematical perspective this only means that the distribution is not uniform. Whether this is a negative thing is another question.
What could be the reason for so few women to decide to pursue a scientific career in AI?
I would say that it is because of the more general underrepresentation of women in science/STEM areas. However, I am not sure about the reason behind it (at least in Europe). As for the statistics of bachelor studies in computer science at Jagiellonian University in Kraków, the proportions are quite similar – i.e., nearly 50-50 in terms of gender. However, this number changes drastically when moving to Master and PhD studies. And those levels of education are usually considered important while pursuing a career in AI/ML.
From a broader perspective, the public opinion on what an AI/ML scientist is, and how to become one, what to study at college, etc. is very inadequate to the truth. In consequence, I believe when young people come to the point at which they are supposed to decide what to do in the future (e.g., choose science studies, programming, etc.), they are horribly misinformed. The kind of mathematics/physics/computer science taught at high schools is in my opinion completely different from what you actually learn at university.
I remember that during high school, when people heard that I enjoyed drawing, painting and writing, or that I played violin, they often came to wonder why I chose a mathematical-oriented class. “You have a creative, artistic mind”, they used to say, “not a scientific one.” I did not know a good answer to that comment back then. Today I know that science is probably the most abstract area of work I have ever come across, and that creativity, especially in research, is an inseparable part of it.
What could change the situation of women in IT research?
We should better inform children at every level of education what it means to be a scientist and how diverse their tasks can be, and why it is important to study science. At least in Poland, standard education makes it very hard to enjoy math or physics and it’s very hard for kids to understand the beauty behind them.
It is not enough to address promotion programs, internships or scholarships at the college level, since this is already too late. We should act and promote women in science from the kindergarten. We really lack the representation of diverse female role models in mainstream media and pop culture (beside Maria Sklodowska Curie). I’ve read about a study which asked preschoolers to draw pictures of a professor and a scientist. All of them drew a mad male scientist. This means that some bias about who can be a scientist is already present at a very early age.
I am not sure about the girls-only internships or scholarships, or science schools and programs in which you predetermined how many men and women are you going to hire. I do understand that the goal is to get more women into the AI/ML field, but sometimes I feel like “you think I need it, because I would not be good enough to beat men in this scholarship application as well?”, or “OK, so I just got that internship because I am a girl?”. This is extremely hurtful. I am every bit an AI scientist. I do not want to get a job because I am a girl, I want to get it because I am good at science.
If you were to point out one character trait that describes a good scientist, what would it be?
Persistence.
What is the interest in AI/ML among students and PhD students in Poland? Are there any related cultural factors that can encourage women to pursuing a career in AI/ML or discourage them?
Interest in AI/ML is very high among computer science students. Not sure about other areas of science. When it comes to cultural factors – I think that in Poland we still often, perhaps not even consciously, assign certain roles to men and women. For instance, women need to take care of children, make dinners and clean. Men have to provide for the family, take care of the home and technical matters, and do manual labor. Obviously, we rarely say such things to each other, but I think subconsciously we still have these divisions in our behavior. So, women go to work, but they still might be expected to do all of the above-mentioned duties at home.
Imagine coming back from work. Are you still cooking dinner, or cleaning the dishes after someone left them in the sink? I would often do this, because I used to feel that this was expected from me – which probably is also my problem, but I think a lot of people feel that way.
Finally, having a family is also an issue. Firstly, PhD studies are not the best way to earn a living for the family. Secondly, you cannot really afford to have one year off because of the pregnancy. The employer and/or university will support you, but you cannot hope for the AI/ML community to stand by and wait. And even a few months break in AI is enough to completely change the state of research.
