03 March 2009

Please test Jaunty...soon

Jaunty's release is less than two months away. Please grab a Jaunty alpha 5 or daily live cd and test! Go through the bugs you've filed and see if they still exist on the live cd. See if any new ones pop up. And do it ASAP.

In every development cycle there's a huge influx of bugs in the last two weeks because a lot of people jump from stable to beta or RC, skipping alpha completely. Great, we've got more testers, but…how many bugs can be fixed in two weeks without risking breaking anything else? Not nearly as many as are filed. That is why we need more people testing early on.

I'm not saying everybody should install the development release on their only computer or the one they use all the time, directly on the hardware. While I've been doing that for the last two years, that's generally not recommended. If you cannot function pretty much entirely from the command line, don't put it on your hardware. Put it in a VM.

VMs aren't terribly useful for testing hardware, but there are plenty of GNOME and KDE bugs you can find using a VM. Try it! Check out Virtualbox. Put Jaunty in there and play with it. Try doing anything that doesn't involve important persistent data in the VM. By that I mean: don't write your thesis in the VM without having it backed up elsewhere. Web browsing? Getting your email over IMAP? Chatting? Why not?

But for hardware, use a live cd. Very often a triager asked "is this still a problem in the current development version?" The end of that sentence is "or would it be a waste of time for us to try to hunt it down when it's already fixed?" though it's not said. Also very often, the bug reporter will reply "I don't know. I'll test it when it releases." Insert whatever development cycle you want in there. It happens in all of them. Guess what happens when the reporter says they'll test after release?

  • 10 The bug gets ignored as "potentially fixed; waiting on feedback" because as I said, trying to fix what's already fixed is a waste of time.
  • 20 The development release becomes a stable release.
  • 30 The reporter complains that it's still not fixed.
  • 40 Work starts on the next development release.
  • 50 The reporter is asked "is it fixed in this one?"
  • 60 The reporter doesn't answer or says again that they'll test after that release.
  • GOTO 10

Bugs can't be fixed without information. Thus, bugs that have a lot of information are more likely to be fixed. If you're not giving us any information, what do you expect?


Tretle said...

It can be irritating filling bugs some times. Like when the bug reporter goes through the trouble of providing all the information needed and assigns it to where it should be assigned and then someone assigns it to something else and then asks what the bug was attached to. Instead of just reassigning bugs here there and everywhere the person that resigns the bug should be responsible for taking down any information they are changing like the original assignment.
I am guilty of not responding to questions on bugs but this is mostly through to this case of providing information which gets wiped and a month or two later having people ask you questions which you already provided the answers to, this is a big no no but it happens to 50% of the bugs I file.
Just thought I should say this here because its not always a case of someone being too lazy to respond. Filing bugs should be as quick and easy as possible and there shouldn't be a case where you have to come back and provide the same information again, its like going through a drive through and having them repetitively ask you your order because the machine is broken, its going to make you less likely to go back. :(

Jonathan said...

Yup, I've been doing this for the past couple of release cycles (both testing a livecd on hardware and lots more software testing in VBox). I haven't tested my hardware yet this cycle though, so thanks for the reminder.

On another note, I realize this might not be the best place to ask this, but do you know if there is any ongoing effort to fix webcam support in Jaunty? Intrepid broke a lot of hardware support for people's webcams. The latest versions of GSPCA kernel modules have fixed a lot of outstanding issues, but the fixes don't appear present in Jaunty, where things are still broken. This seems like kind of a big usability issue that was introduced with (ironically) BetterWebcamSupport from Fedora. It's frustrating for a user to have their webcam not work, and it's especially frustrating when the drivers are actually available. Is this something where we just have to wait for a newer kernel (and thus 9.10) or can these fixes be pulled into 9.04? Bug report here: https://bugs.launchpad.net/bugs/293259 and other related bugs.

Mackenzie said...

I don't think the reporters are lazy. I think they're (rightly) afraid of running alpha versions. I'm trying to present ways to do it safely. The "Activity log" link in the upper right of a bug report will tell you what has changed, though I don't know how to fix the "unobservant triager" problem.

Gensys Logic webcam support is coming, though I know the video support is slow (photo is much better). I can't get my other webcam at /dev/video1 to work though. Haven't determined if video1 v. video0 is the problem yet. Will have to test with other laptop.

Jonathan said...

Do you know if there's any way for me to help promote getting an updated version of GSPCA into Jaunty, which seems to fix a number of people's problems using a variety of webcams?

Mackenzie said...

Feature Freeze is in effect, but if you'd like to discuss fixing such regressions with the kernel team, please bring it up on the kernel-team mailing list.

Stoffe said...

On the other side of the coin, there's a LOT of "triagers" that only see triaging as a game of numbers, and will run through copious amounts of bugs pasting boilerplate about "have anything changed?" and then closing or otherwise changing the status to something that is guaranteed to not get it fixed. These people seem to a) think closed bugs means fixed bugs, and that the release gets better by having the number of open as low as possible in launchpad (false) and b) filling 5-a-day as soon as possible means you're doing a great job (also false).

If these people, instead of pasting useless questions that often clearly shows they haven't even read the actual report would try to test and reproduce a small number, or even one of these bugs themselves instead, they would do actual good instead of racking up their "high score". There was a developer just the other day on the planet saying much the same, though from the other point of view.

When I say that they aren't even reading the bug reports, I am not kidding. An example would be asking that kind of question about packages that has seen no change in version in Ubuntu nor Debian between versions. The odds that something else in the distro has changed so that the bug is fixed is are pretty low...

I want to try and help checking my own bugs, I really do. It's just not always feasible for a number of reasons, and the last thing I want to hear then is some drive-by "triager" that just closes my very valid bugs (usually with some rude boiler plate about how it's closed because noone cares) because that "triager" thinks it's all about racking up points.

A lot of it is about the attitude, really. It makes you stop filing every bug, because it's not handled seriously anyway. Ah well, at least it's better than trying to file *anything* in the Gimp bugzilla, where i suspect automated scripts close them with WONTFIX. ;)

Mackenzie said...

I didn't check my RSS feeds til after I posted this, but I did see that post.

Yeah, that's a problem. Maybe we need to improve the wiki documentation? I'll go add a bit about using rmadison to check and see if the version has even changed since the bug was filed.

Mackenzie said...

Is this an improvement? Note that the __ things make that chunk of text be underlined.

Stoffe said...

Yes, definitely good improvements. :) I'm not sure about the hard line at 4 weeks at all, I would like to see another prod really, but i understand that there needs to be some process in place.

I think that the other thing that could be improved is the understanding of the purpose of this work (and with that, approach and attitude).

For instance, I like the idea of 5-a-day, but on the other hand it easily creates an environment where some people will try to get on the "high score tables", as such actually exists. Wanting to be there is not a bad thing in itself, but I think some people easily lose focus on what the purpose of doing this work at all is. An unfortunate and unwanted effect.

Perhaps that could be addressed as well somehow? Especially when talking about any points system such as 5-a-day (which wouldn't be so "bad" without the high scores) or launchpad karma or similar.

Mackenzie said...

Yeah, I thought the line was 60 or 90 days. 4 weeks surprised me when I saw it on there. Looking at last June, it seems it has said 4 weeks for at least that long. It sounds pretty short to me though. I'd rather post a reminder comment at that point so the reporter gets an email reminding them that someone's waiting on them. I'm going to ask Brian Murray what he thinks about making reminder comments part of policy.

Andreas said...

I try to do that, I often test live cds of alpha versions etc. and file bugreports or comment on them (in launchpad or on kde.org for example) but it's frustrating when nothing is happening. I'm willing to respond to questions but there aren't any pretty often. For example there is this bug that makes Ubuntu unusable for me on my netbook (https://bugs.launchpad.net/ubuntu/+bug/297263), but nothing has happened yet. :(

Mackenzie said...

more changes to Bugs/Responses

I'll send an email to the Ubuntu QA list. After 4 weeks, we'll send reminder emails. If no response within two weeks of the reminder, it can be closed.

Honestly, there are just plain a LOT of bugs, and we're not necessarily going to stumble around onto all of them. It doesn't really hurt to go into #ubuntu-bugs and ask if someone can take a look. Often real-time debugging in there can move things along much more quickly. And those lurking in the channel will learn more about what things to check. Everybody wins!

Andreas said...

Okay, that's understandable, but it's still frustrating. ;)
I didn't know about that channel, so thanks, I will try it that way next time too! :)

Tretle said...

An example of stoffe's point of people not reading bugs. https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-nv/+bug/315357

Mackenzie said...

By the way, the 5-a-day charts are gone. At last UDS they talked about having the charts be "days in a row where 5+ bugs were touched" instead of letting people do 50+ on some days.

Mackenzie said...

That says it was automated. I think it was sent out to all X bugs which lacked attachments. By the way, it's generally preferable for such things to be attachments so we don't have to scroll to Atlanta and back.

Stoffe said...

More good changes, thanks! That's the way I want it, too, with a reminder (or two, hehe).

The charts may be gone, but I saw this post on the planet today, with what I think is well-meant but sometimes encouraging suboptimal behavior: http://blog.qa.ubuntu.com/node/34

Specifically: "Wanna be famous? Is easy! remember to use 5-A-day so if you do a good
work your name could be listed at the top 5-A-Day Contributors in the
Ubuntu Hall of Fame page!"

The heart is in the right place, but pushing the "fame" button like that may have side-effects as per above. :)

Mike said...

meaning-less rant...

As a bug reporter, I wonder what I should do if i was told to retest the problem on a jaunty-alpha-liveCD, but the live cd could not boot(it got stuck on the screen where it asking if you want to install or other stuff). I don't want to burn multiple live cd's for each alpha; I only have so many cd-r's. I can't install the alpha since I _need_ the computer to function. if I report the problem with the live cd not booting I will be expected to test the next livecd-alpha; I will have to burn a lot of cd's. So I have not reported this bug since i know it will just be closed since i won't test the up-coming alpha's. I REALLY want to help test but I keep finding things making it harder on me. I don't want to burn a bunch of alpha-live cd's. Is there a way to boot from a cd-image that is on disk with out buring it? I tried to make start-up disk with my thumb-drive but my computer's BIOS can't boot usb-drives(arr i think there is a way around this with GRUB but I am still frustrated I can't test any thing).

guptaxpn said...

How do you file bugs?

I'd be more active if I was being directed on an easy-to-follow/easy-to-implement bug reporting howto/system.

Thank you.

James Gray said...

Mike - virtual box can boot a CD from the .iso file. No need to burn. However, there is no telling if your bug will still showup under virtualization. If it doesn't
you'll be able to test other stuff. :-)

All -
I am downloading the alternate alpha 5 for ADM as I write this. Sometimes I feel like there are so many bugs in Ubuntu it isn't worth looking for more. Most of what I find is already reported and when it isn't often I discover the problem is an issue with the way I've configured my system.

I'll give it another shot though. :-)

As far as 5-a-day goes: At one point my boss asked everyone to resolve
(not just touch) one bug a day and the result was a significant raise in
the bug submissions within 3 months. I think it is a mistake to reward lots of
work if you don't also reward work based on quality.

Eivind said...

Ubuntu seems unable to handle the bugload they already have.

For example, I reported this: https://bugs.launchpad.net/ubuntu/+source/nautilus-actions/+bug/254171

in august 08. Then I re-tested and found it still present in the next ubuntu. Now I re-tested and it's still in Jaunty.

Indeed there's no indication that anyone has even glanced at it in the last 9 months. Yeah, sure, it's a trivial bug, but frankly, so should the fix be. (trivial, that is)

Testing, writing bug-reports, updating the bug when new releases are available, it's all rather pointless when it has no effect.

I've never had a bug in the kde-bug-tracker sit unchanged for even 3 months. I've pretty much given up reporting bugs to ubuntu. (I do report them upstream though, if they seem likely to be upstreams-bugs)

Mackenzie said...

To report a bug, just get an account at Launchpad. Then go to the Ubuntu bug page. Tell it which package you found the bug in (if you can't figure it out, that's ok). Under the big text box it'll tell you some base iformation to include, such as what version of Ubuntu you found the bug in. Describe what you were doing when you found the bug, what happened, and what you expected to happen. Try to be very specific ("in __ menu, choose ___ option and then....") so it's easier to reproduce.

See what I said above. You're right, there are many more bugs than people to try to take care of them. I'm not actually sure about the bug you linked, since it might affect more than one part of the system.

I definitely encourage everyone here to try to help out with triaging when you've got some spare time. We can always use more help.

Sulamita said...

I'm extremely disappointed with treatment for bug reporting. I have four bug reports in my account, plus two or three that "affects me too", with absolutely no answer.

What's the point on going over and over about the need for providing more information, when this information is never checked once is provided? Let's not even mention the ridiculous absurd of the ubuntu-bug tool, making impossible for a non experienced user - which I though Ubuntu was aimed to be - to use.

Tony Yarusso said...

I tried loading up Alpha 5 in virtualbox, but unfortunately I can not for the life of me figure out how to make it use a resolution higher than 800x600, and you can't even open Evolution at that size. Feel free to enlighten me... (Yes, I tried fiddling with the guest tools stuff.)

Mackenzie said...

:( The Guest Tools stuff "should" work, but I'm actually a KVM user. I didn't want to recommend KVM because it really falls into the PITA category, though it does go for the largest 4:3 resolution it can fit (vertically) automatically.

If you want to try KVM for Ubuntu guests on Ubuntu hosts, python-vm-builder is probably the easiest way.

Mackenzie said...

Also: Evolution doesn't work at 800x600? Ew! Bad! 800x600 is supposed to be GNOME's minimum resolution.

James Gray said...

The easiest way to fix the screen resolution issue may be to install the guest additions. There is a button under devices that *should* mount a virtual CD/iso image containing the guest additions. Launch a terminal and then
cd /media/cdrom
sudo ./VBoxLinuxAdditions-x86.run
(Use the ADM version if you installed the AMD64 version of jaunty). You'll need to restart for the additions to take effect.

Once you have the additions installed you can resize the virtualbox window and the screen resolution of the VM should change with it. Cool!

SeanG said...

Do you know where I can find info on how to install Ubuntu on a RAID 1 setup AND keep my Windows install?

Mackenzie said...

I've never used RAID, sorry. That might be a question for http://www.freelinuxhelpline.net/ They have a weekly live show, and you can call in. The people running the show and those of us hanging out on IRC submit answers to the questions.