Lame Duck Comparison

I am an expert in the Sextractor software. It's a great tool. Not perfect (the coding could be cleaner, better annotated etc etc) but a veriable swiss army knife of a piece of software. So people use it as a yardstick to compare their own code to. However, this often happens in cases where it has already been clearly stated that SE won't work too well (crowded fields for example).

So a paper like this one annoys me slightly:

http://arxiv.org/abs/1211.5805

Probabilistic Catalogs for Crowded Stellar Fields

Brendon J. Brewer, Daniel Foreman-Mackey, David W. Hogg

(Submitted on 25 Nov 2012)

We introduce a probabilistic (Bayesian) method for producing catalogs from images of crowded stellar fields. The method is capable of inferring the number of sources (N) in the image and can also handle the challenges introduced by overlapping sources. The luminosity function of the stars can also be inferred even when the precise luminosity of each star is uncertain. This is in contrast with standard techniques which produce a single catalog, potentially underestimating the uncertainties in any study of the stellar population and discarding information about sources at or below the detection limit. The method is implemented using advanced Markov Chain Monte Carlo (MCMC) techniques including Reversible Jump and Nested Sampling. The computational feasibility of the method is demonstrated on simulated data where the luminosity function of the stars is a broken power-law. The parameters of the luminosity function can be recovered with moderate uncertainties. We compare the results obtained from our method with those obtained from the SExtractor software and find that the latter significantly underestimates the number of stars in the image and leads to incorrect inferences about the luminosity function of the stars.

The issues I have with this paper are twofold. First, the comparison is in crowded fields, an area that every piece of SE documentation (including my own) states is not what to use SE for.
And the actual comparison is extemely weak. They only tweak the detection threshold, which is only one of the relevant parameters here. The other are the level of deblending that could be tweaked and the number of deblending thresholds. I bet I can retrieve quite a few more than 30% of the stars with just SE.

So the whole paper now comes down to making their own (computationally heavy) approach look good. The SE catalog is a straw man. Of course I have not been asked to referee it so they're probably fine.