evaluate.htm: How to search the web, by fravia+: evaluate

~ Evaluating results ~

				Evaluating results

Version March 2000

[Introduction] [Essays] [Sielaff's lessons]

Introduction

You should have learned in the [basic] part of my site how important it is to acquire a systematic approach in order to evaluate the tools you will use for searching, learning what they will cause you to receive - or keep you from receiving - even before starting your queries. Well, that's nothing. It is actually even more important for you to learn a systematic approach when evaluating the (many) documents, clues and results that you will receive as a result of your (I hope well-prepared) queries.

Since I had the honour to have had myself an incredible expert in this field - as teacher - for many years, I'll use a simple 'lore' to let you understand the importance of a correct evaluation approach...

Sielaff's lessons

Old techniques can be very useful for modern evaluation purposes
by fravia+, March 2000

...Yep, I remember Professor Frjthiof Sielaff's lessons about scripts' evaluation lores... let me state that those old lessons are VERY IMPORTANT for anyone that searches the web nowadays... no, I will not "state" anything (it would contradict the very purpose of this very teaching)... just read the following "lore" and judge by yourself...

We sat there waiting and ziemlich gespannt, January, quite cold in Berlin, but, after all, that was the reason we were there: Sielaff was still teaching, against all political odds.
And he came, with his classical quarter of an hour "academic delay"... He came "herein" leaning on one side, because he carried at least a dozen books under his left arm, and then he sat down, without even looking at us, and let all his books fall on the teacher's desk.
Mind you, this was the most cherished post-university course for historians of the early middle ages in Europe in those days, so you can imagine how silent and attentive we all (students from all over Europe) were...
He still did not look at us. "These books" - he began - "deal all, more or less, with the same subject: a history of Danemark in the middle ages. Now please try to supersede the subject: actually the subject could be completely different, it would not matter in the least per se: the point is that you should bei Gelegenheit learn how to EVALUATE all kind of books BEFORE buying and/or reading them (to do it afterwards is not easy either, by the way, but at least that kind of evaluation requires more obvious skills)."
Sielaff looked sharply at his audience. "I imagine" - he said - "that you already know that most of the books... and data... around us are next to useless, don't you?"
Dunnow about the others, but this took me by surprise: I had always thought, naïvely enough, that anything that was published must have had some sort of "value".
"This book..." - Sielaff began, taking the first book out of his pile - "...is titled, quite appropriately for our today example: 'History of Danemark in the Middle ages'..." - he paused and looked at us - "...unfortunately, the Author, as stated by himself in the introduction, does not know danish at all, therefore... " - he suddendly threw the book on a far corner of the teaching desk in disgust.

YOU CANNOT WRITE ABOUT SOMETHING WHICH IS LANGUAGE-RELATED IF YOU DO NOT KNOW THAT LANGUAGE
Obvious but very often underestimated

"This book, on the other hand..." - he continued, picking another book from his pile - "... which is titled 'Danemark between 500 and 1200' has been written by an Author that actually does happen to know danish... unfortunately (for him, on the other side quite fortunately for those among us that want to appreciate books like this on the fly) the same Author wrote - before this book - a book titled 'Cactusses and other desert's plants' and, should that not be enough for you, he wrote, immlediately after the book I am helding right now in my hand, another book titled 'Aquarium techniques for home and profit'.

YOU CANNOT WRITE ABOUT UNRELATED THINGS AND PRETEND YOU REALLY KNOW SOMETHING ABOUT ANYONE OF THEM
Obvious but very often underestimated

"I am sure you begin to understand, now..." - he said - "...but let's continue, because there is MUCH more to understand and reverse... here is another book: 'Danemark in the middle ages' - and he fished a third book from his pile - "that has been written by someone that definitely knows danish, that only worked on danish history of the middle ages, and that happens to be a recognized authority in such matters" - he paused and then throw the book disgusted on the "crap" side of his desk - "I know you wont like what I will now say" - he added - "but it is quite important nevertheless: the Author wrote this book when he was only THIRTY years old!" - in fact we were all still approaching thirty ourselves and therefore more or less speechless - "I fear you'll probably don't dig it right now, but believe me: if you want to be really sure someone knows anything about what he's writing about - especially in complex sciences like early medieval history&mbsp;-, you wait until he is AT LEAST fifty years old, and even in that case you should by all means take GREAT care: most of the so-called "experts" are oft enough just releasing hot air bubbles. This hqs nothing to do with danish history, of course... it happens anywhere, anyhow, anyabout..."

YOU CANNOT TRUST ANYONE THAT DID NOT SPEND A RELEVANT AMOUNT OF TIME ON A SINGLE AND SPECIFIC THEME
Obvious but very often underestimated

"You see..." - continued Sielaff - "...the fact that the previous author worked a lot on a single subject may be relevant for the books he wrote / will have written at the END of his career. But this does not guarantee nothing at all about the books he may have written at the beginning. And now we come to this article: 'About mediaeval danemark' which was written by an expert on danish medieval history, who was fifty himself at the time of writing. The article appeared in november 1982 on a university monthly collection..." - silence followed, we were trying to guess - "...ahem, I repeat: it appeared in november on a monthly..." - more silence: we didn't know what to say - "...ok: if you don't know it yet, then you better learn it right now: monthly publications are tricky - especially university ones they actually HAVE to publish their 12 issues in order to survive... and get public money for the following years... yet the quality varies considerably, and though it may be relatively easy to find some sound and interesting material for the first, say, five or six months, you'll have to scratch the bottom of the 'Topf' to be able to fill and publish all 12 issues... therefore, my dear students, everything published 'from september onwards' should be regarderd with suspect... mind you: it does not MEAN that the stuff is crap, it is only LESS PROBABLE that you'll have some outstanding work there..."

YOU SHOULD ALWAYS TAKE INTO ACCOUNT THE ECONOMIC RATIOS BEHIND ANY PUBLISHING EFFORT
Obvious but very often underestimated

to be continued