Okay I have been tentatively dipping my curiosity into machine learning. I’m coming from a bit behind the curve because well… it’s got a learning curve and I have a day job. But I’ve been steady picking things up. But mostly, I have been noting where there are opportunities to check Machine Learning’s work. Maybe later a deeper survey came along and we can check ML predictions using traditional astronomy techniques we understand much better.
And that is where I have been putting some of my recent research. A little while ago I checked the XSAGA catalog, a prediction for objects that are below z<0.03 against the deeper GAMA spectroscopic sample. The overlap is much smaller than the XSAGA sample but it gives us a direct measure as to how well this ML technique did (well done John Wu, it was spot on where you predicted the effectiveness was). More about it in this paper in MNRAS (or astroph here).
So that was fun. But I wanted to talk about a second paper where I did a check of a ML prediction against more traditional astronomy. In this case a Galaxy Zoo comparison.
The Galaxy Zoo Catalogs for the Galaxy And Mass Assembly (GAMA) Survey
[astroph]
This started as a much more modest idea: we have two Galaxy Zoo catalogs on the equatorial GAMA fields, let’s compare the voting and make these catalogs public so they can be used by students. My main motivation was to generate something that could be used as a reference for students using the GAMA catalogs and make them easy to use by adding CATAID column to them.
All well and good and I started comparing voting fractions across both the catalog originally made for the GAMA collaboration, using KiDS imaging and voting from the Galaxy Zoo citizen scientists.
But on closer inspection, the Walmsley+ (2023, astroph) catalog only includes voting fractions! And those are from the ZooBot machine learning algorithm that is trained on early voting and then helps with predictions for the rest of the survey. This is a necessary step as voting on the full survey would take too long. So instead of an A/B test of surveys (KiDS imaging vs DESI), it also became a comparison of Galaxy Zoo voting from people and voting according to people+ZooBot!
Fortunately, the questions had remained the same between the KiDS Galaxy Zoo effort and the DESI Galaxy Zoo (+ZooBot) effort. Well mostly. The question tree looked like this:
The main difference is in the very first question. The DESI voting (and the ZooBot trained on those) seem to vote more for smooth galaxies, than one with features. Why would that be?
The difference is depth: the DESI survey is much shallower than KiDS, by design, to get more of the sky. But that means dim features surrounding a bulge, e.g. a disk with spiral arms, will not be as readily visible.
But once DESI Galaxy Zoo (+ ZooBot) can detect a galaxy with features, the voting agrees with each other! How tightly wound are the spiral arms?
Or how many are there?
Sure there is some variance. That was to be expected. But as long as the volunteers (and ZooBot!) identify features, the follow-up questions agree well enough.
That brought me to the next question, can you use the voting predictions from ZooBot for the shallower DESI data to get a similar result. This was of interest to me since I have had a few undergraduate students work on the KiDS Galaxy Zoo catalog with some intresting results:
The Loneliest Galaxies in the Universe: A GAMA and GalaxyZoo Study on Void Galaxy Morphology
Lori Porter [astroph]
Galaxy And Mass Assembly: galaxy morphology in the green valley, prominent rings, and looser spiral arms
Dominic Smith [astroph]
Galaxy And Mass Assembly: Galaxy Zoo spiral arms and star formation rates
Ren Porter-Temple [astroph]
And that last one seemed like a good cross-check. Can we get Ren’s result but using the DESI Galaxy Zoo voting fractions?
And yes. The results are qualitatively the same. The statistics are a little worse because the DESI does not have as much voting on number of spiral arms as the deeper KiDS (because of the first question difference). But there is a pretty clear rise in stellar mass with the number of spiral arms. And the main conclusion, that the specific star-formation goes down is also recovered: the specific star-formation drops slightly with the number of arms.
For the GAMA fields, this may not be relevant since the statistics in Porter-Temple (2022) were much better. BUT you can redo the experiment perhaps with DESI at scale.
Checking ML work remains critical in my opinion. You can’t just shrug and accept black box results. If there are opportunities to cross-check with a different data-set, I think that is perhaps unexciting but critical science.
Final conclusion: ZooBot works!
