Rating Photographic Lenses

The one thing that everybody buying a lens wants to know is "how good is it"? It's a simple question, but unfortunately it has no simple answer.

Some people assume that all lenses can be given a numerical rating which will establish a ranking where each lens is better than the lens below it - but not as good as the lens above it. If this was possible, it would be very convenient - but such a rating system isn't really possible for a number of reasons.

  1. There are many different parameters which would potentially factor into a numerical rating of photographic lenses. For example resolution, contrast, vignetting, chromatic aberration, distortion and color accuracy. On top of this, many of these factors will vary as a function of position in the frame. Image quality tends to be higher in the center and lower at the edges and corners. So you have perhaps 10 or more factors which would have to be folded into the numerical rating. A lens which was very sharp in the center and soft at the edges might then have the same averaged numerical rating as a lens that was uniformly quite sharp across the whole frame. Similarly a zoom lens that was soft wide open at the wide end but sharp stopped down at the tele end might have the same "average" sharpness as one which had "medium" sharpness at all settings. Though these lenses might end up with the same numerical rating, they would perform quite differently at any given setting.
  2. Even if you could come up with some sort of numerical average of all the performance factors which reflected "lens quality", it's quite likely that there would be a variation from lens to lens. One sample might rate at 6.6 out of 10, while another might rate 6.3 and a third 6.8. Since pretty much every lens test I've ever seen reported was single test done on a single sample, it's simply not possible to say which of two lenses with similar ratings is "better". If you don't believe this, just take a look at several different sources (websites and magazines). You'll find many examples where lens "A" will be rated higher then lens "B" by one source, yet lens "B" may be rated higher than lens "A" by another. They can't both be right for the "average" sample of that lens - but they could both be right for the individual lenses they tested! You can of course average user ratings of lenses, but the problem then is differing standards. One user might rate a given lens as at 6 while another may rate it at 8, because of their different needs, experience and expectations.
  3. Currently there are tests being done using APS-C frame Digital SLRs, Full frame Digital SLRs and 35mm film SLRs, as well as tests done using scientific methods based on MTF determination using point and line spread functions. All these different testing techniques may (and will) produce different data. In particular the testing of full frame lenses on APS-C DSLRs will result in significantly better edge sharpness ratings and lower amounts of distortion, vignetting and chromatic aberration than similar test would on a full frame camera.

 

In reality I think the best you can do is to rate lenses on a "fuzzy" descriptive scale. Perhaps ratings of Excellent, Very Good, Good, Above Average, Average, Below Average and Poor. Even then the difference between lenses on the low end of "Good" and the high end of "Average" might, for all practical purposes, be pretty much equal in performance and a lens that rates "Average" for full frame performance could well rate "Good" on an APS-C DSLR. Confusion, confusion! I've seen some sites which quote ratings to two decimal places. 6.48 and 6.53 for example. Clearly such precision does not mean that the numbers (whatever they represent) are accurate to that degree. 6.48 and 6.53 might give the false impression that these are very accurate and precise ratings, but in fact they're both really about 6.5 and in fact what they most likely mean is that both lenses probably "rate" somewhere between about 6 and about 7.

The lens ratings assigned on this website are derived from an analysis of many sources. Magazine reviews, web reviews, user comments and, in the case of some lenses, my own testing and user experience. I've gone through as much data as I can find and derived what I think is the best consensus based on the testing methodology and what I think is the reliability of the data. I've used a 1 to 10 scale only for the purpose of sorting lenses by "quality" on an approximate basis. 1 point difference may not be significant, but a 2 point difference is and a 3 point difference certainly is. So lenses with ratings of 9 and 10 are both very, very good and may give quite similar image quality, but a lens rated as 7 is certainly not going to give an image with the same technical quality as a lens rated at 9 or 10.

Finally, for most photographers, absolute technical image quality is just one factor in deciding which lenses to purchase. Also to be factored would be "usability" (zooms tend to be more convenient than primes, when handheld, Image Stabilized lenses will often give superior image quality to non-Image Stabilized lenses, even when the latter are optically "better"). Another factor is compatibility. What are the chances the lens may not work properly on your next camera even if they work fine on your current camera? Yet another factor is focus speed and accuracy. The world's sharpest lens won't be very usable is focus is slow and not dependable.

[Incidentally, the world's sharpest lenses are the process lenses used for semiconductor photolithography. They can be diffraction limited at UV wavelengths and have a flat field the size of a semiconductor wafer - up to 8" in diamter. They can transfer features as small as 100nm (0.1 microns), which corresponds to 10,000 lines/mm. In photographic optics, a good 50mm macro lens stopped down to f4 or f5.6 is about as sharp as lenses get, probably resolving 300-400 lp/mm]

So take "ratings" with a pinch of salt, a sceptical frame of mind and realize they're just one factor to be weighed in deciding which lens is right for you. They're certainly useful but you shouldn't place absolute faith in them, no matter which website or magazine is their source - even if it's this one!

 

Rating here are given in the lens listings on a scale of 1-10 based mainly on optical performance. If there's some uncertainty is whether a lens should be, say, a 7 or an 8, then other factors like construction quality could push the rating one way or the other. Here are the approximate meanings of the numerical ratings:

"Pixel Peepers" (those who view their images at 100% on a video monitor looking for problems) would probably be best sticking to lenses rated 10, 9 and maybe 8. Photographers who actually make prints and view them from a reasonable distance would most likely still be happy with lenses rated at 7 or above, maybe even 6 at times (if the lens was used stopped down). Casual photographers who mostly make 4x6 prints to put in their photo album would likely be quite happy with lenses rated at 6 and above, and might even be happy with some lenses with a 5 ratings. I probably wouldn't recommend anyone buy a lens rated at 5 or less unless it was really cheap and the user was pretty uncritical. Remember though that lots of people use disposable 35mm film cameras with 1 and 2 element plastic lenses, which would probably rate somewhere around a 4 on this scale!