The blog hasn't died, really it hasn't

I know, I haven't updated this blog in ages. I figured I'd make my first update in such a long while an update about some of the projects I've been working on, either on my own or with Tracy:

• Fixing up the diacritics problem in the Classical Music Library and Contemporary World Music MARC records. There are still no diacritics (I'm not sure if would be possible for our OPAC to display those diacritics, anyway), but at least the stuff that looked like gibberish and was unsearchable and unreadable by human beings has been dealt with. Tracy and I worked for an hour and a half on this problem ("Diacritics Hell"). Neither one of us had any idea it would take quite so long.
• I'm still working on adding 505 fields (contents notes) to records. Right now, I'm concentrating on PS647-PS648 (in our General Stacks area). We've got records for anthologies in this area that don't say what titles or authors they include, meaning that the only way someone would know that, say, History Revisited has a story by Harry Turtledove is through some source that is not our catalog (the record for this book now has a contents note, by the way).
• The records to which I'm adding 505s get other special treatments, which is why I only do a few per day, or there wouldn't be any time to do anything else. To list a few things I do: add our holdings to OCLC if they're not already there, fix up the title control numbers, and add genre/form headings (which are actually just LC subject headings used in 655 fields - for now, until there are more genre authority records and I'm actually able to load those authority records, this seems to be the best way to go).
• I'm cleaning up 505 notes that already exist in our catalog. Some of them are enhanced when they shouldn't be, which creates false hits in our title searches. Some of them are unenhanced even though they should be enhanced, which means things that should come up in a title or author search don't. I can fix it globally, but the fix is imperfect, so I have to at least glance at the records before I reload them, slowing the process down a bit.
• I'm fixing 245 fields (Title) that have subfields in an incorrect order. The incorrect order creates display weirdness that sometimes makes it incredibly difficult to figure out what the actual title is, or see that it's the audiobook version instead of the print version. So far, I've done our vinyl records and VHS tapes, which probably took care of the worst offenders.
• I'm still loading authority records in order to authorize our headings and make it easier to keep them up-to-date. It's a slow process that I will never finish.

I think that's it, not including all the projects that have ended up on the back burner.

Electronic theses and dissertations

I added 115 URLs and their corresponding e-resource item records to bibliographic records for theses and dissertations today. In theory, every thesis and dissertation for which we have a bibliographic record and that is available via Proquest should now be searchable as Type: Thesis/Dissertation, Location: Online Access. Yay!

While I was at it, I also cleaned up some stray issues in the records, and added abstracts to records I was editing that did not already have them.

Newest project, with an observation

Now that I've finished the project that flipped subfield d's and q's into their correct order and got rid of all or most of our obsolete subfield w's, I've got yet another project on the table. I realized that, even though I can't export records based on publication year, I can export them based on record creation date, which should usually be within a year or two of the publication date. With this is mind, I'm hoping to add and (where necessary) enhance contents notes in bibliographic records added to our catalog in the past five years. All of the below is being done using MARCEdit, by the way.

So far, the first step is going well. I exported the records, extracted all the ones that have 505 fields (contents notes) with " / " in them, extracted all the ones in that file that don't have 520 fields (summary, etc. notes), and globally enhanced all the 505s. However, not all of these 505s really need enhancing, and there's some potential for error in how enhancing occurred, so I'm going through the records one-by-one before reloading. This is still going a lot faster than enhancing them all individually would have gone.

I haven't had much of a chance to work on this project, because of all the end-of-semester stuff that's been happening, and because cataloging new things needs more attention right now. Still, I've looked through enough records to discover something I hadn't realized: our NetLibrary records, which I had always assumed were the highest quality ones, sometimes have contents notes that end prematurely. The contents notes might only cover half the actual contents of the book, with no indication (via the first indicator) that these fields are in any way incomplete. I'm fixing them up as I come across them, but it makes me wonder what other kinds of problems there might be that I don't know about.

Current big global editing project

I'm halfway through my current large global editing project that is cleaning up the name headings (flipping subfield q and d so that they're in the correct order), deleting obsolete subfield w's in access points, and fixing obsolete indicators in several fields (100, 700, 110, 710, 260) in our oldest records. I looked at the numbers, and I think it'll take 10 more days of work to finish the whole project up. Not bad.

After this project is done, I think I'll go back to concentrating more on straightening up our authority records and name and subject headings - a never-ending job.

While I was doing some subfield q and d flipping, it occurred to me that the technique I was using could be used to fix other problems we have. Since the technique took a bit of work and a lot of testing for me to figure out in the first place, and since every step must be done in a particular order, I decided to save myself future pain by posting instructions, complete with screenshots, in our staff wiki. That'll keep me from having the reinvent the wheel when I finally get around to doing those other fixes.

More deduping, plus an explanation of why it is necessary

I did 6 or 7 more deduplication tests using MARCEdit, with no true success, but a little minor success. On the plus side, I can produce an overzealous list of duplicate records that includes true duplicates and a few that only look like duplicates (for example, same title, but one is a newer edition than the other). That at least gives us a list to work from, I suppose, although matching on ISBN would give a more accurate and probably more complete list.

In case you're wondering (I know this and my last post are somewhat technical), duplicate records are records that are basically for the exact same title - it was published by the same publisher, published on the same year, etc. When we get e-book record files from vendors, we sometimes get records for the same title from multiple vendors. Some of the vendors have records with OCLC numbers in them, some don't, and sometimes they might have OCLC numbers in them but not the same ones that another vendor used (yes, OCLC has duplicate records, lots and lots of them). When we load them, we end up with multiple records for basically the same thing. Ideally, we'd like to have an e-book that is available from multiple vendors accessible on one record.

That's where record deduplication comes in. Right now, we could do our deduping by searching each and every e-book title in the catalog and clearing up duplicates as we come across them. This is not a good idea - we have tens of thousands of e-books, and the number will only grow. The tests I've been doing are part of an attempt to automate deduplication, or at least come up with a list of potential duplicates so that we could avoid having to search every single title in our e-book collection.

I think I'm going to start reading articles on record deduplication. I probably should have done this earlier - if I find something right away that could help us, I'm going to kick myself.

Trying to dedupe records...and failing

While provider neutral e-book records are a nice idea, it's a little hard to do in practice when you're dealing with vendor e-book record packages. Today will be Round 3 of me trying to figure out how to dedupe our records without having to go through each title one by one.

In theory, deduplication could be done at the record loading stage, using, for instance, ISBNs as a second match point. In practice, this probably wouldn't go well, unless we decided to have print and electronic formats on one record - by matching on ISBN, we would end up matching our e-book records with our print records. There are probably other issues with this method that I haven't even thought of. I could do some testing, but I haven't really focused much on this method of record deduplication yet.

Instead, I've mostly been looking at methods of deduplication using MARCEdit. The obvious method, using MARCEdit's deduplication tool and trying to dedupe on ISBNs, has so far failed. I'm either using the tool wrong, or it's not working the way it should. The first day I started experimenting, I remember having some success by matching on main title information. I think I might try that again today. Unfortunately, that would result in multiple editions of one title being considered dupes. If it also lists actual dupes, it would still be better than nothing. Instead of having to search hundreds of titles, maybe we'd only have to search a few dozen. Or so I hope...

Vacation, catalog maintenance

Wow, it's been almost a month and a half since my last post.  My vacation had a little to do with that, but the rest was just...pre-vacation near burn-out, maybe?

My vacation went great. It took me a while to get comfortable with my niece, since I've never really been around babies before, but now I find I feel sad that I won't get to see her very often. At the very least, everyone in her family but her mom and dad is going to miss out on her first birthday - so sad!

Being back at work feels a little weird, but that'll wear off. With SCUUG only a week away, I've been reminding myself how to use MARCEdit for catalog maintenance by working on a project I started looking into right before my vacation. An unknown number of name headings in our catalog are messed up, with subfield d coming before subfield q, rather than after. I had been ignoring this problem, but now it's starting to interfere pretty significantly with my batch authority searching and loading process.

An example of the problem:
Babcock, C. J. $d 1894- $q (Clarence Joseph),

Should be:
Bacock, C. J. $q (Clarence Joseph), $d 1894-

In the past, I occasionally fixed these by hand as I came across them. However, this is annoying, and also bad for my wrist. Global editing is a good thing, and this looked like something that should be fixable globally. I just wasn't sure how.

It turns out it's possible with MARCEdit, and I figured out how to do it all on my own. Woohoo! I'm planning on running the fix for all the oldest records in our catalog (nearly 200,000 I think) over the course of a few weeks. That should take care of most, if not all, of the problem, and then I can get back to batch searching and loading authority records.

While playing with all of that, I also learned the first few steps for a new tool in MARCEdit that allows you to extract certain records from a larger file, edit the smaller file of records, and (in theory) re-insert the edited records back into the larger file. This will be great for all kinds of projects, once I figure out how to get the reinsertion part to work.

Unauthorized subject headings

I've managed to reduce our unauthorized 650 fields (subject headings that don't have matching authority records in our system - this doesn't necessarily mean they aren't valid subject headings) to below 16,000. Or maybe that's the number of bibliographic records that contain unauthorized subject headings, I'm not sure. The number does not include Killeen-only records (which will one day be disappearing) or Cross Timbers stuff (which use subject headings in ways I don't quite understand and am not sure I should be messing with).

It's a small victory, and it's a long way from 0, but I'm still happy about it.

Pages and pages of names

For a while there, I had been working on batch loads of authority files for names in our 100 fields that WorkFlows said were unauthorized (which still leaves the 600s and 700s, and maybe 800s). I sort of stopped working on that project for a while - I think that I was working on the C's when I stopped. I thought I'd done a pretty thorough job, and then I started working on the project again. From A to C, I was able to batch load more than 2500 authority records. I think a lot of them came from e-resource records, which means every time we get batches of e-book records added to the system, the number of unauthorized headings I have to deal with just balloons. Really, I already knew that, I had just never had it demonstrated to me quite so clearly before. Oh, free MARC records. Even the best of them bloat my catalog maintenance reports.

I'll keep plugging along. Going through all the names to find the ones that shouldn't be batch loaded is annoying, though, like a word search puzzle that never ends. Luckily, I may be getting a little help with that. We'll see.

I've got my procedures for name headings pretty much figured out, but I need to take another look at the way I've been doing subject headings. Hmm.... On the plus side, I'm getting really good at doing nifty things with Find and Replace in Word.

Item IDs transformed into barcodes

I dug up Merry Bortz's "Report Makeovers" PowerPoint presentation, in which, among other things, she explains how to turn item IDs in reports into scanable barcodes using MS Word's Find and Replace - and I actually got it to work! The current cleanup project I applied this to is fairly small - only 60 or 70 item types need to be globally changed - but it's nice to know that I've figured out how to do this, in case I've ever got something a bit larger and/or more complex to deal with.

That was probably one of the most awesome SIRSI presentations I've been to so far, and I'm applying what I learned on an almost weekly basis - maybe not the "item IDs into barcodes" part, but definitely the parts where she talked about cleaning up reports. That's how I set things up for batch searching in OCLC.

Some presentations are just so worth the entire cost of a conference - and I'm saying this 6 months after the conference. :o)

World War II is over - next up, the Vitenam War

Technically, we're dealing with the Vietnamese Conflict, not the Vietnam War. Hopefully, all our Vietnamese Conflict headings are now updated - I was going to check, but I seem to be having problems accessing the library's website. More than likely I'll have some manual cleanup to do, though. World War II took more manual cleanup than I expected. Just a few examples - headings in which the war started in 1930 or 1936 (typos, don't you love them?), and an instance where the war was a geographic subject heading (which would mean World War II was a place, not an event - an interesting idea for, say, a writer to play around with, but not something that has any place in our catalog).

World War II ends...TONIGHT!

Or so I hope. Last night I discovered to my embarrassment (as I was helping a student, actually) that we still have books with subject headings in which World War II hasn't ended - the subject heading used is "World war, 1939-" instead of "World war, 1939-1945." I'm hoping they'll all "flip" to the correct heading tonight.

This is why ongoing catalog maintenance is a good idea. I've also noticed that, if a person has died sometime after 1950, there's a good chance they're still alive in our name and subject headings. "Aged" (which should be "Older people") is a known problem and being worked on - I could flip them overnight, but there tend to be more problems with these records than just a single out-of-date heading, so I'm doing them by hand. Headings containing "Afro-American" (which should be "African American") are also still in the process of being flipped - the remaining ones basically need to be done by hand.

::sigh:: I could really use a second me for full-time maintenance work. However, our "browse" searches are working much better than they used to.

Authority control, without the shoulder pain

For most people at DSL, the biggest benefit of all those unsmooshed fields we now have in our catalog is that subject and name headings can now be used properly. For me, the biggest benefit is that I can now run reports that will allow me to do some of our authority work in batches. The process still takes time, and there are so very many headings that will need to be dealt with individually, but being able to do any of this in batch makes me happy. Yesterday, I started writing up procedures for how the reports need to be run and massaged so that batch searching with them is possible, but I've already loaded 10,000 or so authority records with the help of this method, so I know it works.

By the way, prior to our unsmooshing project, those 10,000 authority records would have probably taken me more than 4 months to go through and load, assuming that the resulting shoulder pain from moving my hand back and forth between my mouse and keyboard for each record didn't completely cripple me. I ♥ batch searching.

Status of the "unsmooshing project" - Finished!

As of 12:36 PM today, I loaded what I think are the last of the records with "smooshed" headings and fields. I say "I think" because 1) it turns out there were a few stragglers I almost missed because mine and Tracy's method of extracting them by call number didn't catch them (invalid call numbers? strange Sudoc numbers? I saved the records and plan on checking or sending a student to check) and 2) it's possible that there are records with smooshed fields that don't fit the criteria we set up when we extracted them.

Even if we missed some, we still managed to touch and fix an amazing number of records in slightly less than two weeks, and almost 24,000 access points that were previously unusable for anything other than keyword searching are now usable as actual headings (i.e., they will now display correctly in browse searches, you have a reasonable chance of finding more items with the same heading when you click on them, etc.). Nice.

My next step will be to start running all kinds of maintenance reports, now that they won't be cluttered up with all of that smooshed junk.

One step closer to fixing the "smooshed" fields!

I emailed the MarcEdit listserv for advice and now have a regular expression that I can modify that will hopefully be able to take care of a large chunk of the "smooshed" fields in our older records. Very, very nice, plus I plan on taking this regular expression apart as part of a project to learn more about MarcEdit's regular expressions. I've used regular expressions in Microsoft Word (they're called wildcards there) and have done some nifty things, so I'm excited about the prospect of learning how to do similar things in MarcEdit.

Now I just have to carve out the time for all of that "playing." All the webinars last week and the time I spent figuring out how to globally change the type on our thesis and dissertation records lost me most of the small lead I managed to gain over the Processing students, so it may be a while before I'll have the time to do a lot.

Large print books

Cathy W. had a question for me yesterday (or maybe today, my memory is like Swiss cheese) that brought something to my attention - there's no way to find large print books using our OPAC, or even, as far as I know, a report in WorkFlows. You can limit a search in WorldCat just to large print books that we own, but it can't generate a list of every large print book we own. The best option right now: I can come up with what is probably an almost complete list of them using OCLC Connexion. Since this list is based on the master records available via Connexion, rather than our own local records, it doesn't include our call numbers. It relies heavily on the assumption that the previous catalogers who added our holdings to those records were paying attention to the little code in the fixed fields that can indicate whether something is large print - it's quite possible that these items we have that are supposedly large print really aren't, and I've already seen that we have a few books that are probably large print but that aren't coded as such in the master records. The biggest drawback, as far as DSL's users and people working at the reference desk are concerned, is that the only one who can generate this list is someone with access to Connexion.

There's a way to fix this, however. There's a subject heading, "Large type books," which I could add to every large print book record we've got (it'd probably be more correct to add it as a local genre heading, but, without a genre index, that would defeat the purpose of creating something that people can actually make use of). The "tidy" way to do this would be to take a look at everything in our holdings that Connexion told me was large print (34 items, as of this morning) and confirm that it really is large print. The faster way to do this would be to just add the subject heading to all 34 records, plus the few stragglers that seem to have miscoded master records - some of them might not actually be large print books, but the project would take up less of my time.

It's not really a "rush" project, so I'm thinking about it for now.

Darn - a half-day of doubt

Warning: somewhat long and whiny post.

The retreat today was fun and informative, probably the best part about my day today (not counting my dinner, which will be delicious :D). Unfortunately, aside from that I feel like one of those cartoons where there's a character being followed around by a dark, rainy cloud. Just one of those days, you know?

Aside from being generally confused about the status of the small cart of things I decided could be weeded yesterday (if only I had procrastinated just a bit longer!), my cataloging today was filled with doubt. It wasn't any particular thing that inspired my doubt - just general doubt. Things like:

• It's the Library of Congress's practice to only record "ill." and/or "map(s)" (for the most part), choosing not to specifically say when those illustrations are actually photographs or portraits or music or charts, etc. On AUTOCAT, not too long ago, a lot of catalogers complained about this lack of specificity and talked about how it could actually be helpful to users to be more specific. Those catalogers are more specific. I follow LC's practice, because I decided I have better things to do with my time than figure out what the various abbreviations and terms are for the different kinds of illustrations. Plus, when I flip through, all I need to pay attention to is whether there any pictures or maps at all, not what any of them actually look like. Sometimes I wonder if this is a mistake, and if it would actually matter to people whether an illustration is a drawing or a photograph, or whether a book has portraits. I have no idea what the catalogers before me did, or if they even thought about it at all.
  
• Why do I record/check information in records that people aren't going to see and/or that isn't searchable? The answer to this is that, if it's not recorded/checked now, then if, in the future, it does become searchable (even if only by librarians via reports) and/or visible to the public, the library, our users, our collection, and the Cataloger (me or whoever) will suffer the consequences. Every time I work on catalog maintenance, I see the results of information not being recorded or checked (for whatever reason) - that's part of why I'm such a perfectionist (some might say "anal") about various record details. I understand all of that, and yet there are days when it's a wee bit depressing to enter or check information when I know it has no immediate use.
• Also, leapfrogging on the whole perfectionism thing, should I care less? If "quantity cataloged" matters more than "quality of the records" (because "quantity," by its nature, is easier to quantify for statistics than "quality") are there things I should be considering no longer doing? The things that take the most time (for book cataloging, anyway) are things like adding new authority records (the things that make sure we have cross-references in our "browse" searches) and adding table of contents information or summaries. DVDs, videotapes, and any original cataloging are also automatically time-consuming. I don't want to "skimp" on things - I wouldn't be doing them if I didn't think they were important - but I think about this kind of thing when I hear about catalogers who've been laid off, or administrators who criticize catalogers for caring too much about the details (isn't that what we're supposed to be doing?). I also think about this kind of stuff when a new project gets added to my "to do" list. Thankfully, as far as the former stuff goes, our library has a pretty "cataloger-friendly" work environment. As far as the latter...ugh.
  
• I don't know if you read this blog, Janie, but you might be interested to know that I flip flop between thinking that multiple formats should be on one record and thinking they should each have their own, separate record. I hate that I can't be ok with one decision or the other.

And that's only a sampling of the things that sometimes make me doubt my cataloging. Even as I think these kinds of thoughts, I keep doing things the way I "should" be doing them, or the way I decided I should be doing them, and I think about possible workarounds for anything where the capabilities of the catalog seem to be failing the library (of course, it's always possible that they're not failings, but rather just things where our settings need to be modified...). Still, I prefer the days when these thoughts aren't so much downers as energizers.

Thoughts while pulling stuff for weeding consideration

I spent most of yesterday pulling stuff to potentially weed and then looking over the stuff I pulled. Aside from the depressing reminder that a lot of the stuff we have in some subjects is very old, the task gave me several thoughts (actually, what really got me thinking about some of this was a great ALCTS e-forum, but weeding reminded me).

Say something hasn't circulated much, and we've had it in our collection for a while - I'm wondering, is the real reason some of this stuff hasn't circulated our catalog records? When I catalog stuff, I now add table of contents notes and/or summary notes to almost everything that doesn't already have that information. You'd be amazed at the number of books that don't have this information - I'd say that books on education, computing, and sports seem to be the worst offenders. It can sometimes take a bit of time, but I think it's worth it to input this information for the added keyword access. However, we have tons of records that don't have this information. We even have lots of records that are basically brief records (I have no idea what the exact numbers are, or even if there's a way to find out) - they have title and author information, but not much else. Those materials are basically invisible, unless someone knows the exact title or author or happens to find them while browsing.

It's wishful thinking, I know, but I'd love to one day upgrade some of those crappy records. One way to start would be to identify sections that should be getting more use than they are, or maybe titles that are really hard to find unless you already know how to find them. When I chose to add to the title access points for the Publication Manual of the American Psychological Association, it was because I knew that it tended to be difficult to find unless you knew how to look for it. I don't know general areas that need that kind of attention, though.

I wonder if there's a good way to find all the brief records in our catalog? The only thing I know about those that separates them from the other records in our catalog is that they tend to be all in caps, but not all records that are all in caps are brief records. Plus, I can't tell our ILS to give me a list of all records like that - it's just not possible to search that way.

Hmm, something for me to think about... Of course, that doesn't mean I'd ever have time to work on all those records. But, if I ever get a student worker, one I could trust to work directly on our records, I could have that person add table of contents information as part of a project.

Known issues

In my office, taped to the TV stand, is a list of what could probably be called known issues or, I should say, issues known to me. I have no way of knowing if anyone else knows they exist or, more importantly, knows their implications for searching our catalog. I've debated whether and how to get the word out. On the one hand, it would be helpful for reference librarians to know that, say, the first series title in our MARC records is probably not the one they should be clicking on, regardless of the fact that it is clickable. On the other hand, some of these problems, like the series title link problem, don't necessarily look like problems. Maybe people are happier not knowing, so they don't have to put up with my attempts to explain what's going on and why it's a problem in a non-jargony way (which I'm not always successful at doing...)?

Explaining in person, one on one, as someone talks to me about a time they had problems with a search that was caused by a problem I know about, seems to work ok. I'm not so sure about group explanations. So, what I'm thinking of doing is turning the "Catalog Records" page in our wiki into a sort of "known issues" page. If the issue comes up, I can point to the page, which I will try to make sure includes "what to do about it" sections for each problem. That doesn't necessarily help much with the problems that don't look like problems, but one step at a time, I guess. I'll just have to make sure that I update the page as the problems get fixed.

Awful Library Books - a blog that encourages weeding

I found out about this particular blog a while ago (but still after Yvonne) - Awful Library Books is hilarious and embarrassing at the same time.

It's hilarious because the books that they blog about are so bad (or so strange). It's embarrassing, and should be for any librarian, because these books are all signs that some library's collection is in desperate need of weeding.

The blog focuses mainly on public libraries, but academic library collections need weeding too. Yes, academic libraries should sometimes keep certain older books, perhaps for their historical value or because they are important in their field. It's not possible or advisable to keep everything, however. If the university has no courses that cover the historical aspects of certain topics (such as plant biotechnology, for instance), why keep books whose only current value is historical? There are lots of subjects that tend not to age well if you're not interested in their historical aspects - anything with technology, medicine, science, etc.

I'm not sure this gets mentioned a lot when the benefits of weeding are discussed, but, as a cataloger, I like the idea of large weeding projects, because that usually means lots of old, maintenance-needing records will be removed. I like catalog maintenance, but I like it even more when I know that the records I'm maintaining represent materials people would actually want to use.