Saturday, June 29, 2019

The cost of forced obsolescence

Two developments in astronomy have focused my attention on the topic of forced obsolescence, a favorite of the tech industry. This has crept into the astronomical world. The first instance is the slew of 1m telescopes that got retired during my PhD years. Of course there is a lifetime to everything and planning a retirement is a good thing.  

Now we are finding ourselves at the point where IRAF and its wrapper pyraf are relegated to obsolescence as well as the Astrophysics Data Service (ADS). Both are effectively forcing the astronomical community to switch over. I have been in two minds about it. 

The switch to python 3 and therefore all things in python and astropy is motivated simply by the fact the python 2 language is being phased out by the python project itself. With that reality it is inevitable. This is causing quite a number of issues since not every observatory and workhorse instrument has a few FTE lying around to re-code and test all their code again. So I think the compete deprecation of IRAF is bad. The astropy project simply has not kept up with replacing the toolset. But at least what’s in there works. Is reasonably tested and pretty well documented. Where to find the right software tool is still a mess but that’s equally true for IRAF. 

A worse example is happening with ADS. This has been the mainstay tool for astronomers to search the literature. Astronomical libraries are gone at most places and google.scholar does not fill the same niche. So it is critical it works. It has worked so far. So well even that no one even thought about it much. Almost too well one could say. 

Enter “better”. The old interface is clunky, the codebase (whatever the f that is) needs to be better. So a new version has been in the works for years. That’s okay. But not the switch is being made. And I have made an honest to god attempt to stay ahead of this and switch early. 
Short version: I don’t trust it. 

Missing results, variable return, unreliable loading, weird non-astronomy results. Let’s take my ADS results. A lot shows up. And I have a pretty unique name. Not just my dad’s physics papers of years gone by (fine, funny, nice to see) but hydrology?! After I mark “astro only”. 

I sat in a cool talk by a PhD student and I wanted to get both her papers. Nope. One was stubbornly missing from the new ADS results. Not that may have been a fluke but enough of this sort of thing and I stop trusting the service. And we are a few months out till it is our only option, the new ADS we have problems with and not trust in. 

But both follow a common pattern. The change is announced, it is often not followed by a good short reasoning as to why and people who are upset about are dismissed as stick in the mud and not willing to innovate. 

What I think is missing is the realization on behalf of the innovators that they are destroying “career capital” for people used to working on the older version. They spend significant amounts of time learning them and now that knowledge is useless. And to do the same thingagain, they’ll need to relearn how. Often in an environment where encouraging faculty to learn a new thing is not encouraged (because no direct benefit to scientific productivity, narrowly defined as paper production). 

This seemingly callous destruction of your career capital is what annoys people the most I think. That and the fact that the trusted thing is now untrustworthy. 

But we gotta.  If only so we do not teach our students obsolescent skills. I’ve switched over to python 3 completely. And I’ll teach myself how to rotate and add images in it. Again. 2min work in IRAF imcalc and imrot. But I need to teach it to my grad student. And we need it to work 3 years from now. 

1 comment:

  1. For IRAF, I would tend to disagree. What is finished is the institutional support by NOAO and STScI, not the usage of IRAF. BTW, factically, there is already no institutional support for IRAF since the last release in 2013. Even the iraf.net forum is a volunteers one only.

    However, IRAF is still there. It now relies on the community to keep it alive. This is the same situation as for a lot of other astronomy software (which is a real problem, but independent of the situation for IRAF). The major difference is that now the people/organizations that want to keep IRAF need to organize this themself (and pay for it, also in form of software developer hours). IMO this is fine: why should NOAO pay for something they don't want to use themself anymore? If your institution relies on IRAF, why shall they not contribute to its support?

    NOAO currently doesn't distribute IRAF anymore themself because of major license problems. Ignoring licensing issues (and not ackowledging other's code authorship) for tens of years is one of the big mistakes they made. It has nothing to do with obsolescense, and you can still get the (cleaned-up) source from the iraf-community web site. Also, IRAF is now included in some Linux distributions (Ubuntu, Mageia, Debian). Nobody is forced to give up using IRAF, and as long as the users organize the support themself, it will not die. If this doesn't happen, it will, ofcourse. But this is not "forced", it is lack of supportive interest.

    Keep also in mind that IRAF *is* a very old and outdated framework, and it is really questionable to put any major efforts into it. So, the only useful point is IMO now to keep it somehow running, and this would not require much community effort. IRAF runs on most current platforms, and it is even prepared when Apple switches to ARM64 at some day.

    Disclaimer: I created https://iraf-community.github.io

    ReplyDelete