There’ve been a few comments on an earlier post here, wishing that LibriVox would add a ratings system to its catalogue, so I thought I’d respond to them (and others, posted over time in the LibriVox forum).

Firstly, it’s TRUE that there are some few recordings that are problematic for various reasons. Some have content which causes certain listeners concern, but I’m not going to bother discussing censorship, I disapprove of it, end of story. However, where it’s down to the audio itself — some folks would like to be able to warn others to avoid these, or perhaps, target them to produce their own (hopefully better) versions. However, it’s an extremely central tenet of LibriVox that ALL readers are welcome. As long as they are able to record themselves audibly and stick to the text, it doesn’t matter about age, gender, accent, ability to ‘do voices’ or even whether they understand the book. And the rest of this post is where I contend that this is not only a Good Thing — but essential to LibriVox’s past and future success.


“But wouldn’t some kind of vetting or rating be a good thing for listeners?” It’s not going to happen, because it’s honestly not that easy. It’s not a question of “make people audition, and then only the ‘good’ people are allowed to record.”

For starters, the standards of measurement could vary a lot (as you can see by the bad reviews of professionally read audio-books.) The audience has a wide and sometimes contradictory range of ideas about what various members of it want to hear.

Then there’s the practical consideration of improvement — if “bad” readers can’t ever start, and therefore don’t get encouragement, feedback, gentle support, then they’ll never become “good” readers. The professionals are paying for coaching, they’re studying at drama school or voice-over classes. What’s available for free? On a global scale? In every language people are interested in reading?

To say LibriVox has NO quality control just isn’t fair — today. Historically, there was no proof-listening phase, and so yes, a number of finished recordings are too quiet to be heard. (And some are likely too loud!) Some have horrible background noise, or reading-stumbles left in. Some of these are being revisited as admins find time and energy. However, the books we are producing NOW have much better quality in these respects. Accented readers are still welcome, people still record with headset / pinhole mics — but technical problems can be picked up earlier and mitigated as best we can. Readers are given help with noise-cleaning, editing, general settings and volume … and if they aren’t able to do it themselves, someone else will polish it up before it’s catalogued. But at the end of the day, it’s the reader’s voice, pace, spoken words — no change in that.


But most of all, avoiding ratings is about supporting people’s self-esteem. And not in a tree-hugging, “isn’t everyone special” kind of way. In practice, very few contributors at LibriVox are professional actors / voice-over artists. They aren’t trained to extract what use they can out of criticism and then let the rest wash over them. If contributions are rated badly, especially early on in a reader’s recording career, they’ll simply stop and find a nicer way to spend their free time. Is that “saving the world from bad recordings”? Yep. So, how many people would be left to make recordings for the world? Impossible to say, but I’d bet it’s a pretty small percentage. Worst of all, it would put off even those who are generally considered “the best”, because few start out that way. Basically, it’s “saving the world from having a large, free audio-book library.” Audible would approve.

There are other online audio-projects who do insist on auditions before allowing people in (or simply record only using vouched-for actors.) They’ve produced a few hundred books, and are often pay-to-download because that’s the only way to subsidise the whole proceeding. LibriVox has completed 3,374 projects, and another 541 are in progress, and that’s been on zero budget, (though, true, the recent donations-drive changes things.) I can’t think of any other audio-projects which work across languages – all the ones I know are mono-lingual. LibriVox encourages people to record projects in 29 different languages (more if you count the contributions to multi-lingual collections.) I firmly believe that the “open doors” policy directly supports this diversity.


I think the main problem LibriVox has, is around educating its listeners. What many listeners want, I suspect, is completely free access to Audible. They’re frustrated because LibriVox is free-but-different (non-pro. voices, mixed voices in books, non-native voices, etc.) They think that just a few little changes, kicking out the very ‘worst’ readers, would fix things. But it doesn’t work like that. A small barrier to entry, however low, would stop all but the most determined voices (or the most pig-headed readers). And that’s not going to benefit anyone.


As an aside, all LibriVox recordings are public domain. There’s nothing to stop anyone setting up librivox-rated.com, linking through to the catalogue, or pulling recordings directly from archive.org. If rating recordings is as essential a part of the listening experience as those who’ve requested the feature over the years tend to claim — it’d be more popular than LV itself – and that’d be fine … we could concentrate on recording and let other people do the thorny decision-making. And of course, the more recordings there are, the more chance there is that some of them ARE “okay”, by the particular standards of any given listener.

This article was filed under About LibriVoxing.

14 Responses to “On the absence of ratings at LibriVox”

Read other people's thoughts (most recent first)

  1. clivecatterall

    Cori

    I disagree that what listeners want is free access to Audible or BBC Audiozone or wherever. There are some real barriers to people accepting the LV approach, most centred on (IMHO) incorrect assumptions.

    1. Amateur recordings will be of poor technical quality. My wife rarely listens to LV recordings because she has tried some in the past which have had level problems, intrusive background noise etc. Once a listener has had problems they may drift away. Fortunately LV PLing now includes checkup on technical quality and the wiki contains lots of good stuff, so I hope these aren’t going to be such a problem. I’ve tried copyediting a small part of the wiki advice pages – do we have any feedback about whether people can understand them?

    2. Inappropriate voice/accent. This is a weirdie. I have read George MacDonald, Mark Twain, Jenny Nimmo and others to my children in my English accent without worrying. Why shoud LV be different.

    3. Poor reading. This genuinely can be offputting. I have struggled through recordings with poor diction, very thick accents, apparently loose false teeth etc. The tree-hugger in me has warmed to many of these readers: one SF reader had such a thick european accent that I had to re-listen to sections of her reading to work out what she had said. By the end of her share of chapters I was disappointed when the voice changed – I had grown to love her reading! However, educating the listener is not likely to be successful. It might be the right thing to do, and listeners may gain more from the readings, but no one is going to want to do something that seems difficult. Whatever we may feel about the ethos of LV, some listeners are going to be put off. I honestly believe that the vast majority of readers – even those who are just starting out – will produce recordings that will be perfectly acceptable. What we need to address is the *perception* that the quality of reading will be poor.

    This can be a particular difficulty with multiple voice recordings. It has often been said that all you have to do is download the first chapter of a book and see if you get on with it. This is no good for multiple voice recordings.

    I agree that rating readers and recordings is hopeless, for the reasons you write and more – every recording (even a solo voice) is a collaboration between readers and the author. How can we separate the enjoyment of the book from the enjoyment of the reading. A good reader can breathe life into a poorly written text: just listen to some of the pulp SF in our catalogue – utter pulp fiction drivel read beautifully (I can’t get enough).

    Maybe the best we can do is take a leaf out of the iTunes store and have a button next to each chapter in the catalogue page that plays a random 20 second excerpt of the chapter. You could check out the “quality” of each reader before you download. I know these are free downloads, but folks might think “what if I get to the last chapter and I can’t understand what the reader is saying”. It’s a bit like the last page of a library book being missing. The fact that you didn’t have to pay to take the book out from the library doesn’t make it any less annoying. The listener has made an investment in listening to the recording and wants the payoff of getting to the end of the story.

    I wonder whether users could build up a list of favourite readers. Maybe they could receive a PM when anyone on their list catalogues a new recording. This wouldn’t help the people who seem to feel the need to “protect” the public from “bad readers”, but this sort of thing would be nice to help keep a handle on what is going on at LV. There are so many projects that it can be hard to keep in touch with all the good stuff. M

    If you like genre fiction, perhaps you could receive an email or PM when there is a new addition to that genre. (You’re going to tell me we can do this already, aren’t you…)

    I’m sorry, I’ve strayed a bit. Maybe I should send this as a PM rather than a comment on your blog.

    Clive

  2. Ingrid King

    Regarding Librivox Readers and Ratings:

    Thanks for your post on the question of ratings and Librivox.
    It has given me a new perspective on this issue.

    I am a huge fan of the Librivox: my enjoyment of time spent in my garden or in other activities that leave my mind free to wand has been that much greater with the addition of the wonderful audiobooks offered.

    There are readers who without question greatly add to my enjoyment:
    - by the use of appropriate inflections;
    - effortlessly pronouncing all the written word accurately (which makes the author’s meaning crystal clear);
    - some use distinctive voice styles to represent certain characters;
    - not to mention the talent of pacing a reading just right – knowing where to insert a pregnant pause, how to provide a flow of speech that creates a seamless flow whereby you start to forget that you’re even “listening” to an audiobook – the experience begins to feel as though it’s organically playing out in your mind. I need hardly say how incredibly enjoyable this is :-)
    - by speaking in a “natural” way that allows the listener to pay attention to the text, not the reader.

    I will have to make a point of thanking “Chip” in Florida and “Gordon” is Shropshire for providing this sort of fabulous of listening experience.

    Every so often, however, while fully engrossed in some wonderful tale (Return of the Native by Thomas Hardy at the moment), along comes a reader who is so horribly distracting in their manner, and so utterly mispronounces words, that I inevitably loose track of the story thread entirely.

    This experience occurs to me at least 3 to 5 times per audiobook, and sometimes more often than that.

    Here’s the thing: the 2 readers who spring instantly to mind both have a strange, affected quality to their readings. Rather than using a natural speaking style, they use strange, discordant tones, weird and inappropriate inflections, string out words – hanging endlessly onto the last syllable.
    Every time I encounter these readers I am struck all over again by how unnatural their reading style is. Also, by the end of a reading segment, these oddities sometimes lessen, convincing me further that they may well be affecting their strange reading style, instead of speaking naturally.
    It also happens that the 2 readers I’m reminded of just now both mispronounce words regularly – that is of course not by choice, but rather trying to “make do” as they read the text. It is of course a problem in that it distracts greatly from the author’s intention (meaning and flow of text).
    I suppose this last point argues for reader’s taking the time to pre-read the text and to learn the words they do not know ahead of recording it.

    So I have to wonder if there wouldn’t be some value in feedback. I’m not suggesting ratings, since rating numbers embody finite, judgmental quality, and seem to imply everyone ought to reach some “ideal” reading style. This is of course unnecessary as having a variety of interesting, competent readers is wonderful & democratic, too.

    I’m a believer in “anything worth doing is worth doing well”.
    I am of the opinion that it’s fair to assume that any mature adult can handle nonjudgmental, factual feedback on their work.

    If someone takes the interest and time to do a voluntary reading, I feel you can assume they are motivated to do it well. Why not provide them with the appropriate feedback so they can be confident knowing that they are doing their very best?

  3. Cori

    Thankyou for taking so much time to write, Ingrid, it’s lovely to hear from you.

    It’s funny, I had to look up a few words today myself: “farrago”, “superfluity”, “anodyne” — the problem for me is that I know perfectly well what they all mean, and so when I look at the text on the page, it simply doesn’t occur to me that I don’t know for sure how to pronounce them. So I regularly get into the middle of a recording, as I did with all of these (recorded a month ago now), find I’m unsure — and have a few guesses at the pronunciation, reasoning I can check online later and pick whatever came closest. Interestingly, quite often there’s a couple of alternatives given, particularly with English vs. American pronunciations, but also with loan-words that have been Anglicised over time. Of course this doesn’t account for all mispronunciations, but more than one might think. So simply ‘learning the pronunciation’ isn’t as straightforward as it might sound.

    Even more complicated is this business of feedback, though. You write “I am of the opinion that it’s fair to assume that any mature adult can handle nonjudgmental, factual feedback on their work.” I disagree almost completely. It sounds so reasonable, and yet — almost no-one joins LibriVox as an experienced voice artist. Which means they’re often insecure and uncertain about their readings, and even true feedback, however kindly meant, can kill off any urge to make themselves so very vulnerable in this way. We do offer technical help to boost quality of recording, which can help, and sometimes a little bit of a nudge about speed if someone is particularly fast doesn’t go amiss … but that needs to be among a sea of simple acceptance and gratitude for the time and energy that every reader has put into their recordings. Many people improve over time, but if they’re told right from the start that they MUST improve, that they aren’t okay as things stand, why on earth bother? LibriVoxing is a hobby, a pastime, and there are many others to choose from that don’t involve such a risk. Some people do not improve over time. I often don’t think I’ve changed very much at all. Would the world be better off without my recordings in it? I’m most partial, of course, but would say clearly not. If they aren’t to people’s taste (and they really aren’t, in some cases!) then there’s no law that the listener has to finish the book. But, if I didn’t read those books, perhaps no-one else would. Although people who advocate ratings / feedback (in much stronger terms than you’ve used, Ingrid) think they are “saving the world from bad audiobooks” — it wouldn’t result in the difference between a well-read audiobook and a badly-read one. It’s far more often going to be the difference between an audiobook and no audiobook.

    Also, as soon as one dampens down some readers because they aren’t good enough, other readers self-censor. They think “I’m not a man, I can’t record Frankenstein.” “I’m not British, I can’t record Sherlock Holmes.” They think “perhaps people just aren’t being as honest with me as they are with that other reader. Perhaps I’m terrible and don’t know it.” And that self-doubt stops them dead and that is always a terrible loss because it’s often the most sensitive readers who will hurry down that mental path.

    I genuinely think we’ve done something rather magical with the no-criticism policy. I wouldn’t have predicted it myself … it seems to make more sense to ask for quality (though I wouldn’t be recording myself if that were in place.) And yet, as it stands, we have an amazing catalogue, in dozens of languages, with the ability to choose among several readers for the most popular books, and it keeps growing organically. I don’t think it’s lightly changed and even though I don’t quite understand the source of the magic, I’d resist any change to the policy very strongly (as you can see from this impassioned rant ;)

    Think of it as Wikipedia vs. Encyclopedia authors. It makes no sense at all that a bunch of amateurs can produce anything near as good quality as a thoughtfully-written article by a qualified writer. And yet, given enough people and enough time, it works. It’s an imperfect metaphor because each writer can improve on other’s work in a way that doesn’t translate to recordings … but in terms of widely granting “permission to try”, and ensuring the lowest possible “barrier to entry”, it makes sense. I hope.

  4. Alan

    This position is nonsense. Ratings may be precise, but they’re much better than nothing. The listeners know what’s awful and what’s fair even if the Admins don’t (or won’t say).

    I just returned from a morning walk during which I endured a pretentious rendering of a chapter of Hard Times. Stumbling through the lines, affected British accents, mispronunciations, it was infuriating.

    Awful readers, whether well meaning or otherwise, should not be rewarded. There should be a disincentive for poor reading. Let the people vote or at least like or dislike. It’s unconscionable to make me listen to hours of content only to find half way in that chapter 19 is read by someone with an impediment or that the second half of the book is stumbled over by a plodding nincompoop. Life’s too short for that.

    Do your patrons the service of letting them tell each other about what you have, even if you won’t. You have a wonderful website. The idea is brilliant and typically, your contributors are adequate – a few are even outstanding. But hiding the bad ones under the pretext of encouraging more readers or not hurting feelings is disingenuous and counterproductive.

  5. Cori

    We’re going to have to differ here, Alan. I find our lack of ratings ingenious and highly productive. By setting an easy-to-pass bar, we allow many excellent contributions which would otherwise not exist because people would self-select out before contributing at all. Including myself. You also assume that there is a single standard for ‘good’ and ‘awful’. But there isn’t. Given the choice, I often prefer to hear a female narrator rather than a male one. Some will choose a narrator with a familiar accent over an unfamiliar one (even where the unfamiliar is more ‘true’ to the story or author.) Many will pick a celebrity name over anyone else. Each will rate recordings according to different criteria.

    We want to record every book in the public domain. In order to do that, we need a range of readers with a range of interests. It’s nice to cover Disraeli as well as Dickens. Austin as well as Austen. We aren’t a commercial audiobook publisher. It doesn’t matter if the memoir by E. Nesbit only has a few thousand downloads, since it can’t be found anywhere else. Financially it’s nonsense. There’s (sadly for me, since I’d love to make a living at it) almost no money involved in this. However, it makes no sense to me to say that it’s better to have no version than a bad one. And that is the eventual outcome of ratings, however innocent they appear for listeners.

    I see it as slightly churlish to criticise something produced only out of the goodness of a narrator’s heart. If I might point out, Audible accepts ratings as well as money. ;) I’d highly recommend Martin Jarvis, but Anton Lesser or Simon Prebble will also have done wonderful versions, not to mention the other readers who I’ve not sampled.

    You can also find reviews of LibriVox books at archive.org and at Goodreads. We don’t forbid them or try to hide or remove them. It’s just never going to be a part of our own site.

  6. Sam

    I find it interesting you would use the word “productive” to describe the result of not requiring any sort of quality control. I can sympathize with your cause, but I think the result is maybe meaningless and a waste of time for the listener, which is whom your users should be the most concerned for. In the middle of War and Peace, there is a sudden shift in readers. It has become obvious that whoever spearheaded the project in book one is now gone (or only makes brief appearances), and those who were well-endowed with much talent and had picked it up afterwards have been struggling to finish it. Half way through, a large variety (never quite the same person) of people began picking up the book, and it became nearly impossible for me to do anything else but to listen intently to every word as if it were a puzzle. And it wouldn’t be so bad if these people either spoke slower or edited their tapes. Even the dramatization was enough to drive me nuts. What was worse was how everyone felt they had to bring their own style to the work — throwing out any concept of cohesion.

    I’ve used LV for to get through large books of fiction because I found it efficient, but it’s obvious to me that LV can not fill this void and it would probably be better for me to buy an audiobook service than use LV from now on.

  7. Cori

    We’re an alternative to other services, not a replacement. If you can afford an audiobook service, then it’s great to support professional narrators, editors and producers financially. To complain about something done solely for the love of the book, no money involved for anyone, seems … slightly harsh.

    Our quality control has evolved over time — we now recruit proof listeners for every project, we have minimum and maximum volume levels, we give more support when people have fierce background noise. Our earliest books didn’t benefit from any of those. But if that was the bar we’d tried to set at the start, we’d never have retained enough people to even attempt War and Peace, let alone the rather higher quality recent recordings. Not to mention that the technology simply wasn’t there to support it. This is going back 8 years, and without professional level (super-expensive!!) software, noise cleaning simply couldn’t be done. Affordable microphones were still pretty shonky. And I had a 256MB MP3 player that I could have one or two books on at a time. ;)

  8. sam

    > 2. Inappropriate voice/accent. This is a weirdie. I have read George MacDonald, Mark Twain, Jenny Nimmo and others to my children in my English accent without worrying. Why shoud LV be different.

    This is so arrogant. Are your children the root of all discernment when it comes to book reading? But it does display the attitude of all LibriVox supporters admirably. To LibriVox people, what they think is all that matters.

    LibriVox can of course do as it pleases, but it will never be anything other than a sideline to professional recordings, for the sole reason that most of the books are unlistenable to most people. Maybe you aren’t one one those. If so, good for you, but you are in the minority.

    There are three real problems. First, people who have such strong accents it is irritating to listen to them (indian people that are barely understandable reading Jane Austen springs to mind). Of course, if you are indian you can probably understand them better than an english reader because you are used to the accent, but for those that are not it is a real problem. Second, books with multiple readers. People read books, and therefore listen to books, to escape from reality. It throws you back into the real world when it’s a completely different person reading to you all of a sudden. Third, inappropriate readers. LibriVox supporters think this is a non issue, but it really isn’t, not to the majority of listeners anyway. An inappropriate reader is, for example, someone with a strong southern US accent reading Swallows and Amazons (I don’t know if that’s even been done, I’m just using it as an example).

    LibriVox cares not a jot about this stuff and will never change it, and so will never be mainstream. It is mainly a club for bored housewives and retired people to do something interesting with their time. If you really want to see what the majority think look up LibriVox on sites other than LibriVox itself and read some of the comments. They will very rarely be complimentary.

    On the other hand, and the reason I’m even here writing this, there are a handful of otherwise unavailable books on LibriVox that are read by one person with a decent accent. Cori is one of those. The fact that I’m on this site proves that.

  9. Laura

    Using librivox for the last 4 years, I have not encountered a truly dreadful reader until now. Definitely don’t want to discourage anyone, but it pretty much ruins the book, and it is a shame that this is the only version available.

    I was wondering, could librivox possibly have an internal anonymous flag system for each recording, where if a reading is dreadful — too fast, or unintelligible that it can be flagged? Also, once flagged, the flagger can be asked to enter an e-mail address if they would be willing to “redo” the recording.

    Once a recording gets a certain number of flags, a moderator might check it, then contact the list of flaggers willing to re-record and tell them to go ahead. The new recording then can be traded out for the old one.

    I would completely be willing to re-record some of this last book I am listening to — the reader is so fast, the text is completely ruined.

    Make sense?

  10. Cori

    Hi Laura. It is difficult when there are problems in collaborative recordings. And it’s good that it’s relatively rare. (Depending on the listener, anyway. Some do seem to have worse luck in picking books than others. Or are less flexible about differing accents, etc.) In general I think the flagging system might be too complicated for LibriVox most of the time. However, it’s fine to submit alternative chapters for books! Mostly that happens where either readers get confused about their chapter claims. And the book coordinator decides to keep both. Or in Abandoned Solos, where whoever picks up the abandoned chapters eventually decides to record their own version of them, rather than making the book a plain duet.

    Just sign up on the forum and send a 1-min test as usual, if you’re not already a reader. Then contact an MC (ideally, the one who MCed the book originally, if you can see they’re still active.) And explain the situation and they can take it from there.

    We do get comments on books from time to time. It’s rare that flaggers are willing to volunteer (and rare as hen’s teeth that they actually go through with it — I’m not sure I’ve ever heard of it happening. Am completely happy if you’d be the first, I don’t think it’s a bad idea at all.) Although, perhaps this is what motivates some soloists. Once they get going on their replacement chapters, they start to think how nice a solo of the whole thing would be … ;)

  11. Laura

    Thank you Cori! Sounds like a plan. I am not an established reader yet, but If it were possible to trade out recordings of certain chapters with this particular project, I think it sounds like a fun holiday project. I have been intending to read for a while now, and have the basic equipment, so this will be a good motivation to jump in!

  12. Cori

    I should say that the newly recorded chapters won’t be replacing the others — but offered as alternatives within the project. :) Good luck with it!

  13. John

    Are you ever going to finish Dante? I can’t finish it without you!

  14. Audiobooks and Amateur Voice-Acting | Cutting The Giggles

    […] Cori. (2010, May 2). On the absence of ratings at LibriVox. To Posterity – and Beyond! [blog]. Retrieved from: http://piratelibrary.com/2010/on-the-absence-of-ratings-at-librivox […]

Leave a Reply

Archives