Tuesday, October 26, 2010

More deduping, plus an explanation of why it is necessary

I did 6 or 7 more deduplication tests using MARCEdit, with no true success, but a little minor success. On the plus side, I can produce an overzealous list of duplicate records that includes true duplicates and a few that only look like duplicates (for example, same title, but one is a newer edition than the other). That at least gives us a list to work from, I suppose, although matching on ISBN would give a more accurate and probably more complete list.

In case you're wondering (I know this and my last post are somewhat technical), duplicate records are records that are basically for the exact same title - it was published by the same publisher, published on the same year, etc. When we get e-book record files from vendors, we sometimes get records for the same title from multiple vendors. Some of the vendors have records with OCLC numbers in them, some don't, and sometimes they might have OCLC numbers in them but not the same ones that another vendor used (yes, OCLC has duplicate records, lots and lots of them). When we load them, we end up with multiple records for basically the same thing. Ideally, we'd like to have an e-book that is available from multiple vendors accessible on one record.

That's where record deduplication comes in. Right now, we could do our deduping by searching each and every e-book title in the catalog and clearing up duplicates as we come across them. This is not a good idea - we have tens of thousands of e-books, and the number will only grow. The tests I've been doing are part of an attempt to automate deduplication, or at least come up with a list of potential duplicates so that we could avoid having to search every single title in our e-book collection.

I think I'm going to start reading articles on record deduplication. I probably should have done this earlier - if I find something right away that could help us, I'm going to kick myself.

Trying to dedupe records...and failing

While provider neutral e-book records are a nice idea, it's a little hard to do in practice when you're dealing with vendor e-book record packages. Today will be Round 3 of me trying to figure out how to dedupe our records without having to go through each title one by one.

In theory, deduplication could be done at the record loading stage, using, for instance, ISBNs as a second match point. In practice, this probably wouldn't go well, unless we decided to have print and electronic formats on one record - by matching on ISBN, we would end up matching our e-book records with our print records. There are probably other issues with this method that I haven't even thought of. I could do some testing, but I haven't really focused much on this method of record deduplication yet.

Instead, I've mostly been looking at methods of deduplication using MARCEdit. The obvious method, using MARCEdit's deduplication tool and trying to dedupe on ISBNs, has so far failed. I'm either using the tool wrong, or it's not working the way it should. The first day I started experimenting, I remember having some success by matching on main title information. I think I might try that again today. Unfortunately, that would result in multiple editions of one title being considered dupes. If it also lists actual dupes, it would still be better than nothing. Instead of having to search hundreds of titles, maybe we'd only have to search a few dozen. Or so I hope...

Friday, October 8, 2010

Vacation, catalog maintenance

Wow, it's been almost a month and a half since my last post.  My vacation had a little to do with that, but the rest was just...pre-vacation near burn-out, maybe?

My vacation went great. It took me a while to get comfortable with my niece, since I've never really been around babies before, but now I find I feel sad that I won't get to see her very often. At the very least, everyone in her family but her mom and dad is going to miss out on her first birthday - so sad!

Being back at work feels a little weird, but that'll wear off. With SCUUG only a week away, I've been reminding myself how to use MARCEdit for catalog maintenance by working on a project I started looking into right before my vacation. An unknown number of name headings in our catalog are messed up, with subfield d coming before subfield q, rather than after. I had been ignoring this problem, but now it's starting to interfere pretty significantly with my batch authority searching and loading process.

An example of the problem:
Babcock, C. J. $d 1894- $q (Clarence Joseph),

Should be:
Bacock, C. J. $q (Clarence Joseph), $d 1894-

In the past, I occasionally fixed these by hand as I came across them. However, this is annoying, and also bad for my wrist. Global editing is a good thing, and this looked like something that should be fixable globally. I just wasn't sure how.

It turns out it's possible with MARCEdit, and I figured out how to do it all on my own. Woohoo! I'm planning on running the fix for all the oldest records in our catalog (nearly 200,000 I think) over the course of a few weeks. That should take care of most, if not all, of the problem, and then I can get back to batch searching and loading authority records.

While playing with all of that, I also learned the first few steps for a new tool in MARCEdit that allows you to extract certain records from a larger file, edit the smaller file of records, and (in theory) re-insert the edited records back into the larger file. This will be great for all kinds of projects, once I figure out how to get the reinsertion part to work.

Tuesday, August 31, 2010

Cataloger humor on Twitter

Turns out I even like Fake AACR2 better than Fake RDA.

RDA and the RDA Toolkit

It's the final day of the RDA Toolkit free trial (I will not call it the "open access period," as this implies things that aren't true), so I decided to write a post about it.

When I originally began hearing about the RDA Toolkit, there were some things about it that I thought sounded kind of exciting. I liked the idea of being able to keyword search RDA. I liked the idea of a mapping between AACR2 and RDA and between MARC and RDA. I also thought the workflows feature sounded like it would be really useful.

Then I got to experience the reality of the RDA Toolkit, and not just what all the webinars had been telling me. My original plan was to try to catalog a few things using the Toolkit, to get a feel for what it would be like to actually use it. I also planned to try out any available Library of Congress RDA Toolkit workflows. It turns out my original plan was a bit ambitious.

I haven't been able to catalog a single thing with the RDA Toolkit. Granted, I haven't been able to use it as much as I would have liked, but the bigger problem is that it is so hard to figure out where to begin. If I take a look at the RDA table of contents, do I see something that helps me easily navigate cataloging a book, versus a DVD, versus a website? No. I see "Recording Attributes of..." (there is no way to get the full chapter title to show, not even by expanding the pane, so it's not until you click into a section of the chapter that you can guess it's actually "Recording Attributes of Manifestations and Items"), "Recording Primary Relationships..." (same problem as the previous chapter), "Recording Relationships to..." (again, same problem), and "Recording Relationships between..." (and again, same problem). Where am I supposed to begin? Even when I can find a logical starting place, if I don't consult other chapters, I won't be able to completely catalog the item.

"But wait!" you with knowledge of cataloging with AACR2 say.  "Isn't it the same with AACR2?" Well, in some ways, yes. Chapter 1 is the primary chapter one works with, but that information needs to be combined with other appropriate chapters, as well as chapters for choosing and building access points. However, with my nice print copy of AACR2, it's fairly easy to see the overall structure and pick out the chapters I need to use. I read many of these chapters from beginning to end when I was first learning to catalog, and now I rarely have to consult any of it. However, when I do need to consult it, it's fairly easy for me to figure out where to go. Each bibliographic description chapter has the same overall structure. There's even an index, for easier access when I can't remember exactly which chapter I need to consult to, say, deal with honorifics.

The RDA Toolkit is supposed to be the best way to use RDA: "...most users agree the preferred way to interact with RDA is online via the RDA..." (Troy Linker, AUTOCAT listserv post, "RDA Toolkit Solo-User Pricing, Double-User Offer, and RDA Print," 4/28/10). I'm not sure I agree with this, and I wonder if those users that supposedly said this qualified their statements with "but even the Toolkit is hard to use."  At least with AACR2, navigating the multiple chapters needed to catalog something is made easier with an index and an overall organization that accepts the physical reality of the items being cataloged (which, yes, has drawbacks when you're dealing with items that are electronic, rather than physical, but wouldn't it have been easier to just overhaul the e-resources portions of AACR2?). With the Toolkit, there is no index. When I asked whether there would be an index for RDA, I believe the answer was something along the lines of "probably not" or maybe "we'll consider it."  While I love the idea of being able to keyword search my cataloging rules, that's only a good option once I know those rules. I could probably do successful keyword searches in AACR2.  Unfortunately, one of the big things about RDA is that it overhauls cataloging vocabulary with FRBR terminology. An index with decent cross references could help clear things up, but, instead, I'm left with keyword searching. I know there are those who figure out which AACR2 rule corresponds to which RDA rule (connections which, as far as I can tell, have not yet been implemented as easy-to-use links in the Toolkit), but isn't RDA supposed to replace AACR2? It's not a good replacement if you find yourself having to consult the "replaced" rules just to use the new ones.

So, part of me wonders if it wouldn't be easier to use a print version of RDA, which would also have the advantage of not requiring a yearly subscription fee. I'd save myself a bit of eyestrain by not having to read through it on a screen (yes, you can print RDA from the RDA Toolkit, but then why not buy it in print in the first place?). However, then I remember the monstrous bulk of the RDA draft. Without an index, I'm not sure even a print version of RDA would be easier to use.

When I was first learning to use AACR2, I wrote in it, and I marked pages with post-it notes.  In the RDA Toolkit, you can create bookmarks, which is nice, but wouldn't it be even better if it were easier to add bookmarked information into a workflow? Maybe I've missed something, but the Workflows portion of the Toolkit, the part that I was most excited about, seems awfully clunky. The Workflows editor has many of the features of something like MS Word, and, as far as I can tell, no features that integrate it really well with RDA. When I bookmark parts of RDA, where is the feature in the RDA workflow creator that allows me to easily add bookmarked information to a workflow?  Yes, I can copy and paste, but I can do that in MS Word. There should be more benefits to using the Workflow creator/editor than just the ability to easily share workflows with others.

"The RDA Toolkit is designed to reach several audiences. The largest audience is of course catalogers, but some of the functions while marginally useful for catalogers were designed with educators, system developers, researchers, and the wider metadata community in mind." -- Troy Linker again, RDA Listserv, "Re: [RDA-L] RDA Toolkit - Schemas," 8/26/10

This is a point that has been made several times, that RDA and the RDA Toolkit are intended to be used by a wider audience than just catalogers. I'm wondering, does ALA Publishing really expect RDA to be adopted outside the library community? If even we have difficulty understanding it and how to apply it, I seriously doubt other communities would want to make the effort to wade through it. By trying to be all things to everyone, RDA has, in my opinion, managed to not really serve anyone. If anyone does use it (which library catalogers may have to, if the Library of Congress adopts it), they will probably not be using the RDA Toolkit. Instead, I imagine most people would rely heavily on a "Concise RDA" of some sort, or RDA cheatsheets. Another possibility is that those who do use the Toolkit will rely primarily on borrowed workflows, either used as is or with local edits. The Library of Congress' RDA workflows, while still a bit daunting, are much easier to use for the actual act of cataloging than the full RDA.

Last I heard, our library will not be subscribing to the RDA Toolkit yet. There's really no point. It's too cumbersome for me to begin using right now, and, anyway, there's not point in using it if it hasn't been officially adopted by the Library of Congress yet. Plus, if I were to switch to using it, I would first need to spend a few hours with Tracy, changing our ILS display settings and adding new MARC fields. For now, I guess I'll wait and see. It's just too bad that ALA Publishing set up such a limited trial period - why not extend the trial until the Library of Congress has actually made its decision? I doubt ours is the only library that will not be subscribing because it's not yet clear whether it would be a waste of money.

Saturday, August 28, 2010

A week or so of Second Life

For a large chunk of time, I sort of forgot about Second Life. To get back in the swing of things in preparation for a meeting, I spent the past week or so exploring. Which is pretty much all I do in Second Life, because I haven't really felt inclined to try to build things. Well, except for that one time I accidentally created a cone, which, to my horror, stayed in the world after I brought it into being. As far as I know, the cone is still there, because I wasn't really sure how to get rid of it. Thankfully, it's tiny and underwater, so maybe no one will ever notice.

Anyway, I took pictures. Here they are.


This place is Chouchou_V. I think it's some band's island. It was empty when I went there, but the notecard I was given when I went to the island indicated that the band sometimes does Second Life concerts here. In this picture, you can see me at the top of some very high stairs, perilously close to falling. It's a pretty, creepy place. It makes me think of a cathedral of bone in a dream world.


This is my avatar in a place called Alirium. It has giant bunnies. It's a little hard to tell, but there's also another avatar with wings a little to my left. She never moved or spoke. Then again, I was a little too intimidated to try to strike up a conversation. I didn't stay here long, because the bunnies kind of freaked me out.


The next time I logged on to Second Life, I was a particle cloud. At work, with a less wonderful video card, I became an egg (I'll see about posting that one sometime - it really is pretty funny). Particle Cloud Me could move around, and apparently other people saw my avatar, and not the particle cloud. Even though I could still do things, I wanted my avatar back. I opened a case with Second Life tech support, waited a few days, heard nothing from them, and eventually figured out how to fix things myself. By the way, I was in a place called Nemo when I took this picture. It's a gorgeous steampunk-style island. Like so many other islands I've been to, it was completely devoid of other avatars.




These three pictures were taken on a strange island called Cheese. Cheese is like an onion of crazy - as you move around, you think you've seen it all until new stuff rezzes into being. I found it fascinating, creepy, and a little horrific. From what I could tell, what powers all this craziness is lots of capitalism. Any building you could go inside was filled to the brim with products you could buy. For example, in the third picture, I am within view of a building called "Ultra Mega Mass Homicide," which sells, if I remember correctly, scary clown skins, and a building called "That Handsome Devil," which also sold stuff (can't remember what). I'm standing on a platform in front of another store, which, among other things, sold dancing Michael Jacksons and dancing Christopher Walkens. It's too bad I only know how to take snapshots and not video - this picture just does not communicate the creepiness of turning around to see a frantically slithering Christopher Walken.

Also, you may have noticed, but my avatar is now black-and-white with wings. After finally figuring out how to not be an egg or a particle cloud, I decided to give my avatar a drastic makeover. This was the result.



I'm pretty sure this place was called Little Kasiopaya. These are not really representative pictures. Basically, from what I could tell, everything worth seeing on this island was enclosed in a giant glowing sphere - I'm standing right outside that sphere in the first picture. In the second picture, I turned around to see what was outside the sphere. As you can see, there was lots of space. Literally. Inside the sphere are various scenes which I think are supposed to depict areas of the galaxy. Or maybe giant jewels. In addition to providing some interesting views, I think this island was also intended as a good make-out spot. I came across one couple doing just that, as well as some "hug" and "cuddle"...things (jewels, spheres, whatever). I know, this probably makes no sense. I just don't have the Second Life vocabulary to describe this kind of stuff, and I didn't take the right pictures.


I am 100% broke in Second Life. I can't even do the dirt cheap fishing that one island offered as its fun activity. Luckily, sight-seeing is free. Petting the dogs at the VKC Dog island was also free. Here's my avatar, petting a doberman. All or most of the dogs here cost at least a few thousand Linden dollars to adopt. I think I'll wait and see if I can find my avatar a free pet.

Tuesday, August 24, 2010

Semantics vs. the Library Catalog

"Not" is a very important word - leave the "not" out of any sentence, and you end up with a very different meaning. However, library catalogs don't necessarily care about that kind of thing, which can create a bit of a disconnect between them and their users.

Take this book I just cataloged: I can't believe it's not fattening! : 0ver 150 ridiculously easy recipes for the super busy by Devin Alexander. You won't ever find this book by searching for I can't believe it's not fattening. You'll either have to search for it with everything in quotes, or (the more amusing option) you'll have to search for I can't believe it's fattening, leaving out the "not."

I've heard that there's maybe one ILS out there that doesn't ask its users to rely on special tricks for getting around the "special Boolean words" issue. Ours is not that ILS.