Hello people of arch-releng,
A while ago Dieter asked if someone could write him a (relatively) simple web-app to get a better view of the state of the auto-generated isos. I tried that and have now come up with something to look at. He asked me to send a mail to this list so we could have a look at it in public, so here it is. It's hosted at: http://www.gitorious.org/ryuslash/archweb
I just branched the archlinux website, since Dieter said he might want it to be a part of it, and it's written in django anyway so it's fairly easy to rip it out and place it in a new project. My code is in the testsresults branch in the afore mentioned repo, to set it up just follow the instructions in the archweb README, I added some fixtures so that things like bootloaders don't have to be entered manually by everyone. The /isotests/ page shows the results that have been entered so far, I tried sticking closely to what Dieter requested. The /isotests/add/ page allows you to add a new result.
I would have liked to put the app in a repo of its own, but I didn't find how I can make the templates load from the app/templates directory, so I moved on to this idea.
I don't have a lot of experience with django, so I could be doing some things in a very silly way, I haven't cleaned up the code I've written at all (I should probably do that soon, I'll get on it tonight), but I hope that this is at least an ok starting point.
I hope I exlpained everything enough, if there's anything I still need to do, fix or change just let me know and I'll get on it as soon as I can.
Greetings, Tom
On Thu, Mar 10, 2011 at 3:43 AM, Tom Willemsen tom.willemsen@archlinux.us wrote:
Hello people of arch-releng,
A while ago Dieter asked if someone could write him a (relatively) simple web-app to get a better view of the state of the auto-generated isos. I tried that and have now come up with something to look at. He asked me to send a mail to this list so we could have a look at it in public, so here it is. It's hosted at: http://www.gitorious.org/ryuslash/archweb
I just branched the archlinux website, since Dieter said he might want it to be a part of it, and it's written in django anyway so it's fairly easy to rip it out and place it in a new project. My code is in the testsresults branch in the afore mentioned repo, to set it up just follow the instructions in the archweb README, I added some fixtures so that things like bootloaders don't have to be entered manually by everyone. The /isotests/ page shows the results that have been entered so far, I tried sticking closely to what Dieter requested. The /isotests/add/ page allows you to add a new result.
I would have liked to put the app in a repo of its own, but I didn't find how I can make the templates load from the app/templates directory, so I moved on to this idea.
I don't have a lot of experience with django, so I could be doing some things in a very silly way, I haven't cleaned up the code I've written at all (I should probably do that soon, I'll get on it tonight), but I hope that this is at least an ok starting point.
I hope I exlpained everything enough, if there's anything I still need to do, fix or change just let me know and I'll get on it as soon as I can.
Nice! I'd be glad to have this hosted on the main site (I'm the maintainer). Overall things look sound; sure there are some style/convention/etc. differences but nothing hard to fix up.
* View page: right now it is making 98 queries (one for each of those max values). Not that this page is probably going to see loads of traffic but we can surely improve upon that. * Add page: Using radio buttons for widgets here would probably be smarter than the dropdowns, otherwise field entry is excruciatingly painful. See the forms documentation for how to change the default widget. * models, Test.ms: totally un-descriptive field name, and you didn't give it a nice field name. Can't this just be "modules"? * models, Test: if you name things boot_type instead of boottype, you'll get nicer default labels on your view and entry screens. Same with the models themselves- Boottype -> BootType. * models, Iso: Likely need more than a date field here. We've double released, etc. It might just be worth having a CharField as no users are going to be entering these anyway, and also having a boolean "active" field or something to determine what actually shows up on the entry and view pages. * models, get_success_test/get_failed_test: these can be abstracted into a common proxy model (http://docs.djangoproject.com/en/1.2/topics/db/models/#proxy-models) superclass, from which all these various ISO options inherit from. * tests.py: yes, we should be writing tests, but at the least, get rid of the default garbage in this file. * admin.py: convention is not to use * imports; I know it works here and makes things cleaner but I've eradicated the codebase of them otherwise. * models.py: "# Create your models here." -> not needed * urls.py: You aren't using info_dict anymore; you have a blank second pattern definition?, and I'd prefer you follow indentation patterns of something like packages/urls.py. * views.py, add: A lot of extra comments. "Create your views here.", all the "form been submitted" trailing stuff. For the return, I've switched to using mostly direct_to_template to avoid some of the boilerplate- see the end of flag() in packages/views.py around line 373 for an example. * views.py:, view: I see some things are _choices, some are _list- why the disparity? * isotest/templates: why is this in a different directory from the standard layout? * Finally, I see 4 + 1 templates, but only two views- are these other ones old, unnecessary, not wired up yet?
Great work! I know I just said a lot but these are mostly minor things, nothing to feel bad about.
-Dan
On Thu, 10 Mar 2011 10:04:24 -0600 Dan McGee dpmcgee@gmail.com wrote:
On Thu, Mar 10, 2011 at 3:43 AM, Tom Willemsen tom.willemsen@archlinux.us wrote:
Hello people of arch-releng,
A while ago Dieter asked if someone could write him a (relatively) simple web-app to get a better view of the state of the auto-generated isos. I tried that and have now come up with something to look at. He asked me to send a mail to this list so we could have a look at it in public, so here it is. It's hosted at: http://www.gitorious.org/ryuslash/archweb
I just branched the archlinux website, since Dieter said he might want it to be a part of it, and it's written in django anyway so it's fairly easy to rip it out and place it in a new project. My code is in the testsresults branch in the afore mentioned repo, to set it up just follow the instructions in the archweb README, I added some fixtures so that things like bootloaders don't have to be entered manually by everyone. The /isotests/ page shows the results that have been entered so far, I tried sticking closely to what Dieter requested. The /isotests/add/ page allows you to add a new result.
I would have liked to put the app in a repo of its own, but I didn't find how I can make the templates load from the app/templates directory, so I moved on to this idea.
I don't have a lot of experience with django, so I could be doing some things in a very silly way, I haven't cleaned up the code I've written at all (I should probably do that soon, I'll get on it tonight), but I hope that this is at least an ok starting point.
I hope I exlpained everything enough, if there's anything I still need to do, fix or change just let me know and I'll get on it as soon as I can.
Nice! I'd be glad to have this hosted on the main site (I'm the maintainer). Overall things look sound; sure there are some style/convention/etc. differences but nothing hard to fix up.
- View page: right now it is making 98 queries (one for each of those
max values). Not that this page is probably going to see loads of traffic but we can surely improve upon that.
- Add page: Using radio buttons for widgets here would probably be
smarter than the dropdowns, otherwise field entry is excruciatingly painful. See the forms documentation for how to change the default widget.
- models, Test.ms: totally un-descriptive field name, and you didn't
give it a nice field name. Can't this just be "modules"?
- models, Test: if you name things boot_type instead of boottype,
you'll get nicer default labels on your view and entry screens. Same with the models themselves- Boottype -> BootType.
- models, Iso: Likely need more than a date field here. We've double
released, etc. It might just be worth having a CharField as no users are going to be entering these anyway, and also having a boolean "active" field or something to determine what actually shows up on the entry and view pages.
- models, get_success_test/get_failed_test: these can be abstracted
into a common proxy model (http://docs.djangoproject.com/en/1.2/topics/db/models/#proxy-models) superclass, from which all these various ISO options inherit from.
- tests.py: yes, we should be writing tests, but at the least, get rid
of the default garbage in this file.
- admin.py: convention is not to use * imports; I know it works here
and makes things cleaner but I've eradicated the codebase of them otherwise.
- models.py: "# Create your models here." -> not needed
- urls.py: You aren't using info_dict anymore; you have a blank second
pattern definition?, and I'd prefer you follow indentation patterns of something like packages/urls.py.
- views.py, add: A lot of extra comments. "Create your views here.",
all the "form been submitted" trailing stuff. For the return, I've switched to using mostly direct_to_template to avoid some of the boilerplate- see the end of flag() in packages/views.py around line 373 for an example.
- views.py:, view: I see some things are _choices, some are _list- why
the disparity?
- isotest/templates: why is this in a different directory from the
standard layout?
- Finally, I see 4 + 1 templates, but only two views- are these other
ones old, unnecessary, not wired up yet?
Great work! I know I just said a lot but these are mostly minor things, nothing to feel bad about.
-Dan
Thanks Tom,
I know nothing about django, so thank you Dan for providing your input there. my notes: * add something like: autocmd BufWritePre *.py :%s/\s+$//e to your .vimrc, it makes sure you won't commit trailing whitespace. (oh right you use emacs right? well, configure it ;-) * there will need to be some kind of "are you human test" on the input form (a simple question/response check might do fine, like "which Linux distro is this about? (just the 4-letter word)") * I would prefer even some sort of authentication. if there was a way to allow people to authenticate with their bbs or archwiki login, or even get cookies from the wiki (does archlinux.org get the cookies from wiki.archlinux.org?) this will be needed to have some credibility for entries, as well as a way to get back to the user if I have any questions about their report. also if we have this, we don't need the 'are you human' test. * I will need to install this either locally (any tips on that?) or maybe we host it on archlinux.org directly (if that seems safe enough, Dan? note that we'll need to update it regularly to test updates, but I could review code/queries - but I have zero django knowledge), when I do that, I can give feedback on the actual features and workflow. * Can you provide a sample static page called "help" or something? When I have a setup that I can test on, I can start writing the contents for that help page. It will be needed anyway for users who want to use the app, and at the same time it can serve as implementation guideline for you.
Dieter
On 10 Mar 18:26, Dieter Plaetinck wrote:
- add something like:
autocmd BufWritePre *.py :%s/\s+$//e to your .vimrc, it makes sure you won't commit trailing whitespace. (oh right you use emacs right? well, configure it ;-)
Yes I should :)
- there will need to be some kind of "are you human test" on the input form (a simple question/response check might do fine, like "which Linux distro is this about? (just the 4-letter word)")
- I would prefer even some sort of authentication. if there was a way to allow people to authenticate with their bbs or archwiki login, or even get cookies from the wiki (does archlinux.org get the cookies from wiki.archlinux.org?) this will be needed to have some credibility for entries, as well as a way to get back to the user if I have any questions about their report. also if we have this, we don't need the 'are you human' test.
That would be pretty cool, I only know it _should_ be possible if we just grab credentials from the bbs/wiki/aur database, but I think that would be beyond the scope of just the app I'm writing. Of course I could look at how (re)captcha works.
Django does have this CSRF thing, that somehow should protect you from cross-domain requests, I actually forgot whether or not I'm using this right now, but if I'm not it's very easy it seems, but I don't know if that is sufficient (probably not, you can still write a script that grabs the necessary info that way and starts posting away).
- I will need to install this either locally (any tips on that?) or maybe we host it on archlinux.org directly (if that seems safe enough, Dan? note that we'll need to update it regularly to test updates, but I could review code/queries - but I have zero django knowledge), when I do that, I can give feedback on the actual features and workflow.
If it's just for testing you can easily git clone it and follow the instructions provided in archweb's README, django has a very nice testing webserver.
If you want it in a separate project so you can host it, that's also fairly easy to do, I think, just let me know.
- Can you provide a sample static page called "help" or something? When I have a setup that I can test on, I can start writing the contents for that help page. It will be needed anyway for users who want to use the app, and at the same time it can serve as implementation guideline for you.
I don't know, for as far as I've understood django just doesn't do static pages, you place them where no django url points to on the web sever and whichever webserver you're running should serve that static page (I'm sure Dan or someone else can explain much clearer, sorry).
But wouldn't that be a good idea for a wiki page that is linked to, maybe?
Tom
On Thu, 10 Mar 2011 19:17:37 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
- there will need to be some kind of "are you human test" on the input form (a simple question/response check might do fine, like "which Linux distro is this about? (just the 4-letter word)")
- I would prefer even some sort of authentication. if there was a way to allow people to authenticate with their bbs or archwiki login, or even get cookies from the wiki (does archlinux.org get the cookies from wiki.archlinux.org?) this will be needed to have some credibility for entries, as well as a way to get back to the user if I have any questions about their report. also if we have this, we don't need the 'are you human' test.
That would be pretty cool, I only know it _should_ be possible if we just grab credentials from the bbs/wiki/aur database, but I think that would be beyond the scope of just the app I'm writing. Of course I could look at how (re)captcha works.
I think this app needs authentication (for webbased form input), there are 2 reasons in my previous mail. Why do you think this app doesn't need authentication? captcha isn't needed if we have authentication.
- I will need to install this either locally (any tips on that?) or maybe we host it on archlinux.org directly (if that seems safe enough, Dan? note that we'll need to update it regularly to test updates, but I could review code/queries - but I have zero django knowledge), when I do that, I can give feedback on the actual features and workflow.
If it's just for testing you can easily git clone it and follow the instructions provided in archweb's README, django has a very nice testing webserver.
I'll try.
If you want it in a separate project so you can host it, that's also fairly easy to do, I think, just let me know.
- Can you provide a sample static page called "help" or something? When I have a setup that I can test on, I can start writing the contents for that help page. It will be needed anyway for users who want to use the app, and at the same time it can serve as implementation guideline for you.
I don't know, for as far as I've understood django just doesn't do static pages, you place them where no django url points to on the web sever and whichever webserver you're running should serve that static page (I'm sure Dan or someone else can explain much clearer, sorry).
AFAIK templates/public/download.html is a static page, maybe looking at that can help you.
But wouldn't that be a good idea for a wiki page that is linked to, maybe?
nah, the help page is tied to this application, so the code for it should go along with the source code of the app. it should also be served with the same style, etc.
Dieter
On Thu, Mar 10, 2011 at 1:24 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
On Thu, 10 Mar 2011 19:17:37 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
- there will need to be some kind of "are you human test"
on the input form (a simple question/response check might do fine, Â like "which Linux distro is this about? (just the 4-letter word)")
- I would prefer even some sort of authentication. Â if there was a way
to allow people to authenticate with their bbs or archwiki login, or  even get cookies from the wiki (does archlinux.org get the cookies  from wiki.archlinux.org?)  this will be needed to have some  credibility for entries, as well as a way to get back to the user if  I have any questions about their report.  also if we have this, we  don't need the 'are you human' test.
That would be pretty cool, I only know it _should_ be possible if we just grab credentials from the bbs/wiki/aur database, but I think that would be beyond the scope of just the app I'm writing. Of course I could look at how (re)captcha works.
I think this app needs authentication (for webbased form input), there are 2 reasons in my previous mail. Â Why do you think this app doesn't need authentication? captcha isn't needed if we have authentication.
Why does this need authentication at all? I'm totally confused on this one.
1. No way this will ever integrate with BBS/Wiki login unless you get single sign-on working for everything anyway, so just forget that idea. That has been a dream feature request for years (https://bugs.archlinux.org/task/10703). If you want integration, then this app should be a a) forum thread or b) wiki page. 2. Credibility- we ask for a name and email address here, so that is something. You'll be able to see bogus results if they come through. I'm not sure how this form is any more likely to result in spam than posts to the ML, and if it does, don't accept forms without comments, etc. There are plenty of ways to avoid garbage.
- I will need to install this either locally (any tips on that?) or
maybe we host it on archlinux.org directly (if that seems safe  enough, Dan? note that we'll need to update it regularly to test  updates, but I could review code/queries - but I have zero django  knowledge), when I do that, I can give feedback on the actual  features and workflow.
If it's just for testing you can easily git clone it and follow the instructions provided in archweb's README, django has a very nice testing webserver.
I'll try.
If you want it in a separate project so you can host it, that's also fairly easy to do, I think, just let me know.
- Can you provide a sample static page called "help" or something?
When I have a setup that I can test on, I can start writing the  contents for that help page.  It will be needed anyway for users who  want to use the app, and at the same time it can serve as  implementation guideline for you.
I don't know, for as far as I've understood django just doesn't do static pages, you place them where no django url points to on the web sever and whichever webserver you're running should serve that static page (I'm sure Dan or someone else can explain much clearer, sorry).
AFAIK templates/public/download.html is a static page, maybe looking at that can help you.
Hardly static, and I hate the damn thing as people ride my ass if I am an hour late updating and deploying it. I have to edit it every time we do a release which is ridiculous, so please please please don't follow this. If anything, and I have done this on other Django projects, we should add a "StaticContent" type model that allows arbitrary admin controlled bits and pieces of content to be displayed on our various pages.
But wouldn't that be a good idea for a wiki page that is linked to, maybe?
nah, the help page is tied to this application, so the code for it should go along with the source code of the app. Â it should also be served with the same style, etc.
-1. This is a website, not a version controlled living document as the help page could be (what code is here?). The wiki accommodates this stuff perfectly. We've moved a lot of our home page links away from static-type pages and over to the wiki for this reason; this one seems no different to me.
Wow, I sound negative- sorry 'bout that, but I'm just trying to share from the (bad) decisions made in the past.
-Dan
On Thu, 10 Mar 2011 13:42:26 -0600 Dan McGee dpmcgee@gmail.com wrote:
On Thu, Mar 10, 2011 at 1:24 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
On Thu, 10 Mar 2011 19:17:37 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
- there will need to be some kind of "are you human test"
on the input form (a simple question/response check might do fine, Â like "which Linux distro is this about? (just the 4-letter word)")
- I would prefer even some sort of authentication. Â if there was a way
to allow people to authenticate with their bbs or archwiki login, or  even get cookies from the wiki (does archlinux.org get the cookies  from wiki.archlinux.org?)  this will be needed to have some  credibility for entries, as well as a way to get back to the user if  I have any questions about their report.  also if we have this, we  don't need the 'are you human' test.
That would be pretty cool, I only know it _should_ be possible if we just grab credentials from the bbs/wiki/aur database, but I think that would be beyond the scope of just the app I'm writing. Of course I could look at how (re)captcha works.
I think this app needs authentication (for webbased form input), there are 2 reasons in my previous mail. Â Why do you think this app doesn't need authentication? captcha isn't needed if we have authentication.
Why does this need authentication at all? I'm totally confused on this one.
- No way this will ever integrate with BBS/Wiki login unless you get
single sign-on working for everything anyway, so just forget that idea. That has been a dream feature request for years (https://bugs.archlinux.org/task/10703). If you want integration, then this app should be a a) forum thread or b) wiki page. 2. Credibility- we ask for a name and email address here, so that is something. You'll be able to see bogus results if they come through. I'm not sure how this form is any more likely to result in spam than posts to the ML, and if it does, don't accept forms without comments, etc. There are plenty of ways to avoid garbage.
okay, so no authentication reuse. then i guess the next best thing is to ask a (nick)name, email address and response to a simple "are you human" question. that should work well enough, except it would allow people to purposely add "everything is okay" reports with a bogus name/ email address, which would cause me to mark iso's as "verified enough" and make an official release of crappy isos. (or the inverse: they can mark as everything as broken). if users would be authenticated I think there would be less chance for abuse.
AFAIK templates/public/download.html is a static page, maybe looking at that can help you.
Hardly static, and I hate the damn thing as people ride my ass if I am an hour late updating and deploying it. I have to edit it every time we do a release which is ridiculous, so please please please don't follow this. If anything, and I have done this on other Django projects, we should add a "StaticContent" type model that allows arbitrary admin controlled bits and pieces of content to be displayed on our various pages.
I think with static you mean "rarely/never gets updated". I meant "not dynamically generated"...
But yeah, I wouldn't want to force you doing work whenever I do a typo fix on the page. OTOH if deploying a small update is more work then 1 simple command, then there's something wrong with our method of deploying. Imho there should be a command that you, I or any other dev can execute to deploy updated archweb code. and the tree should always be in a state that it can be pulled from and put live right away (maybe by using a `production` branch?), and when there are invasive changes, needing schema changes or whatever, these should be brought into `production` branch when they are being enabled live by someone who knows what he's doing. (i.e. you).
However, if that is not something we want (it would for example imply I need push access to the repo) or have time to setup right now, I can live with the next best thing: maintaining the help page on the wiki. (I don't like the idea of managing the page content through the webinterface, I want version control, so let's keep this webapp simple and let's not poorly reinvent a wiki)
Dieter
on http://127.0.0.1:8000/isotests/add/ -> list of iso's is empty (since it's required, you cannot currently add new entries). as we discussed, there should we a way (probably commandline based, so we can put it in a crontab) to add iso entries to the table based on the directories @ http://releng.archlinux.org/isos/ I first thought we could shellscript this, but since django nicely abstracts the database, we should do it with a pyhon commandline program instead.
-> install type: put interactive at the top (as it's the most common) -> what does 'Ms' mean?
the pages should have links to each other so that you can navigate to all pages.
On Thu, Mar 10, 2011 at 3:59 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
on http://127.0.0.1:8000/isotests/add/ -> list of iso's is empty (since it's required, you cannot currently add new entries). as we discussed, there should we a way (probably commandline based, so we can put it in a crontab) to add iso entries to the table based on the directories @ http://releng.archlinux.org/isos/ I first thought we could shellscript this, but since django nicely abstracts the database, we should do it with a pyhon commandline program instead.
Django admin- there is no need to waste time on something that has a built-in web interface that takes 3 seconds to add a new ISO name.
-Dan
On Thu, 10 Mar 2011 16:03:55 -0600 Dan McGee dpmcgee@gmail.com wrote:
On Thu, Mar 10, 2011 at 3:59 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
on http://127.0.0.1:8000/isotests/add/ -> list of iso's is empty (since it's required, you cannot currently add new entries). as we discussed, there should we a way (probably commandline based, so we can put it in a crontab) to add iso entries to the table based on the directories @ http://releng.archlinux.org/isos/ I first thought we could shellscript this, but since django nicely abstracts the database, we should do it with a pyhon commandline program instead.
Django admin- there is no need to waste time on something that has a built-in web interface that takes 3 seconds to add a new ISO name.
okay, you want to spend those 3 seconds every 2/3 days to manually add a new iso name? Be my guest. I surely don't. (remember we're talking about iso's that are being automatically generated every 2/3 days, sometimes daily)
Dieter
On Thu, Mar 10, 2011 at 4:14 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
On Thu, 10 Mar 2011 16:03:55 -0600 Dan McGee dpmcgee@gmail.com wrote:
On Thu, Mar 10, 2011 at 3:59 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
on http://127.0.0.1:8000/isotests/add/ -> list of iso's is empty (since it's required, you cannot currently add new entries). as we discussed, there should we a way (probably commandline based, so we can put it in a crontab) to add iso entries to the table based on the directories @ http://releng.archlinux.org/isos/ I first thought we could shellscript this, but since django nicely abstracts the database, we should do it with a pyhon commandline program instead.
Django admin- there is no need to waste time on something that has a built-in web interface that takes 3 seconds to add a new ISO name.
okay, you want to spend those 3 seconds every 2/3 days to manually add a new iso name? Be my guest. Â I surely don't. (remember we're talking about iso's that are being automatically generated every 2/3 days, sometimes daily)
Aha, then a Django management command works fine for this, we already have several. You are still going to have some issues with that dropdown being far too full of things if they are this continuous, or do you plan on pruning old ISOs so we can just mark those inactive in the list?
-Dan
On 10 Mar 16:24, Dan McGee wrote:
On Thu, Mar 10, 2011 at 4:14 PM, Dieter Plaetinck dieter@plaetinck.be wrote:
Aha, then a Django management command works fine for this, we already have several. You are still going to have some issues with that dropdown being far too full of things if they are this continuous, or do you plan on pruning old ISOs so we can just mark those inactive in the list?
Aha interesting, I'll look at that, if that's ok.
I thought maybe any iso's that come before an official release iso might be best marked as inactive? Maybe then we also need to keep an 'official' field so that can be checked and isos before it might be automatically marked as inactive at that point?
On 10 Mar 22:59, Dieter Plaetinck wrote:
on http://127.0.0.1:8000/isotests/add/
-> install type: put interactive at the top (as it's the most common)
Ok, but that is just some data in the database though
-> what does 'Ms' mean?
Yeah I know, Dan mentioned that as well, I've already added it to my todo list :)
the pages should have links to each other so that you can navigate to all pages.
I didn't do this yet because I don't know how other apps within archweb manage their own menus. And I also didn't want to mess around with archweb's main menu yet since I'm not even sure if the name is ok yet :)
On Fri, 11 Mar 2011 09:11:38 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
On 10 Mar 22:59, Dieter Plaetinck wrote:
the pages should have links to each other so that you can navigate to all pages.
I didn't do this yet because I don't know how other apps within archweb manage their own menus. And I also didn't want to mess around with archweb's main menu yet since I'm not even sure if the name is ok yet :)
"isotests" is a good name imho. well, almost. the app is not only about iso's but also about images for usb/pxe. so maybe "testbuild-feedback" but then that's not really clear it's about releng stuff, and "releng-testbuild-feedback" is a bit verbose. maybe "RTF" (acronym for the latter) ?, maybe you guys can come up with something good.
On Fri, 11 Mar 2011 09:08:16 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
On 10 Mar 16:24, Dan McGee wrote:
Aha, then a Django management command works fine for this, we already have several. You are still going to have some issues with that dropdown being far too full of things if they are this continuous, or do you plan on pruning old ISOs so we can just mark those inactive in the list?
Aha interesting, I'll look at that, if that's ok.
I thought maybe any iso's that come before an official release iso might be best marked as inactive? Maybe then we also need to keep an 'official' field so that can be checked and isos before it might be automatically marked as inactive at that point?
The only thing we need is a way to limit the list of iso's on the "add" page to current ones. (like, only display the most recent 20). If you also want a page where you just give a plain list of reports, you should limit it to the 100 most recent or something. Anything else (explicit marking as inactive, pruning old reports,...) sounds like YAGNI to me.
On Fri, 11 Mar 2011 09:24:41 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
On 10 Mar 22:01, Dieter Plaetinck wrote:
okay, so no authentication reuse. then i guess the next best thing is to ask a (nick)name, email address and response to a simple "are you human" question. that should work well enough, except it would allow people to purposely add "everything is okay" reports with a bogus name/ email address, which would cause me to mark iso's as "verified enough" and make an official release of crappy isos. (or the inverse: they can mark as everything as broken). if users would be authenticated I think there would be less chance for abuse.
So how would we do that? a question like which arch is best? or should I start looking at captcha or something?
Any question which can be easily answered by a person who wants to add a report, but not by a spam bot. captchas are generally considered unpleasant by visitors, regular questions are more acceptable. (and they are probably easier for you as well)
Another idea might be (although might cause other problems of course, and would require more effort on the user's part) that results have to be verified, like forum logins and such. As I think that everyone who takes the time to use a web interface to enter these questions might already be motivated enough to also respond to a simple single-click verification, perhaps. And I don't think many people would test a single iso all that many times, at most they might enter a new result if they changed their settings from last time. But it still might not be the nicest idea.
"must be verified, like forum logins" => this is called authentication. But I'm not sure how exactly you want to authenticate them. "simple single-click verification" ? How would that work?
However, if that is not something we want (it would for example imply I need push access to the repo) or have time to setup right now, I can live with the next best thing: maintaining the help page on the wiki. (I don't like the idea of managing the page content through the webinterface, I want version control, so let's keep this webapp simple and let's not poorly reinvent a wiki)
So what is it then you have against the wiki exactly? just the fact that it's managed through a web interface? Since I thought the wiki was version controlled anyway (looking at the history tab sure gives me an idea much like it) and I thought it would be the logical place for any arch-related documentation? I don't want to be a pain I just don't follow completely :)
Yes, a wiki is version controlled, you must have misunderstood me. I prefer maintaining documentation in git and "agile"-style deployments (where I can easily update the code on the server in a reliable way) If that's not possible (or too much work to setup), then a wiki is the next best thing. "next best", and not "best" because git >> mediawiki versioning, cli/vim >> webinterface and I prefer the page to be integrated (both visually, and by its url) with the app. Maintaining this on the wiki is not my preference, but I can live with it.
Dieter
On 11 Mar 18:25, Dieter Plaetinck wrote:
"isotests" is a good name imho. well, almost. the app is not only about iso's but also about images for usb/pxe. so maybe "testbuild-feedback" but then that's not really clear it's about releng stuff, and "releng-testbuild-feedback" is a bit verbose. maybe "RTF" (acronym for the latter) ?, maybe you guys can come up with something good.
Well isotests is the best I can come up with, I think. I kinda like rtf personally.
The only thing we need is a way to limit the list of iso's on the "add" page to current ones. (like, only display the most recent 20). If you also want a page where you just give a plain list of reports, you should limit it to the 100 most recent or something. Anything else (explicit marking as inactive, pruning old reports,...) sounds like YAGNI to me.
That sounds reasonable.
Any question which can be easily answered by a person who wants to add a report, but not by a spam bot. captchas are generally considered unpleasant by visitors, regular questions are more acceptable. (and they are probably easier for you as well)
Yes probably, but maybe not as interesting. But you're right, people probably don't like captchas (I don't really like them).
"must be verified, like forum logins" => this is called authentication. But I'm not sure how exactly you want to authenticate them. "simple single-click verification" ? How would that work?
No I only mean like how forums and other kinds of websites verify email addresses. You send an email to the address they sent and the test results they have entered don't become valid until they click the link in the email to prove that they might actually exist. It was just an idea and not one I thought users might be thrilled about, but I'm guessing not many spambots will respond to such things and I'm also guessing people will get easily annoyed and won't bother this way. Single click was just me trying to explain the process: User enters data, server sends email to entered address, user clicks link in email and server knows the user is more likely to be a real person trying to help improve the community. I dunno, probably not a good idea...
Yes, a wiki is version controlled, you must have misunderstood me.
Yeah, sorry, that's why I asked :-)
I prefer maintaining documentation in git and "agile"-style deployments (where I can easily update the code on the server in a reliable way) If that's not possible (or too much work to setup), then a wiki is the next best thing. "next best", and not "best" because git >> mediawiki versioning, cli/vim >> webinterface and I prefer the page to be integrated (both visually, and by its url) with the app. Maintaining this on the wiki is not my preference, but I can live with it.
Ok got it, thanks.
Tom
On Sat, 12 Mar 2011 00:13:11 +0100 Tom Willemsen tom.willemsen@archlinux.us wrote:
"must be verified, like forum logins" => this is called authentication. But I'm not sure how exactly you want to authenticate them. "simple single-click verification" ? How would that work?
No I only mean like how forums and other kinds of websites verify email addresses. You send an email to the address they sent and the test results they have entered don't become valid until they click the link in the email to prove that they might actually exist. It was just an idea and not one I thought users might be thrilled about, but I'm guessing not many spambots will respond to such things and I'm also guessing people will get easily annoyed and won't bother this way. Single click was just me trying to explain the process: User enters data, server sends email to entered address, user clicks link in email and server knows the user is more likely to be a real person trying to help improve the community. I dunno, probably not a good idea...
This is almost a user registration/authentication system. (the difference is that with the latter, you confirm your email-address only once, and from then on authenticate with a password) Even though it would introduce yet another different account on users, I'm afraid this will be the only option if we want some credibility of reports. (which I find a must for this app) Django probably has an easy way to do registration and authentication.
Dieter
Hey guys,
I didn't get a lot done today, but I did get something done. I've written that django management command to get the available isos, iso's can now be fetched by running `./manage.py sync_isos`
I had some questions as well:
- there will need to be some kind of "are you human test" on the input form (a simple question/response check might do fine, like "which Linux distro is this about? (just the 4-letter word)")
I notices in packages/views.py in the FlagForm class that you are using a textfield to determine you're not getting spambot entires by requiring it remains empty and keeping users from seeing it, would this also be adequate? Seems clever and easy.
- Can you provide a sample static page called "help" or something? When I have a setup that I can test on, I can start writing the contents for that help page. It will be needed anyway for users who want to use the app, and at the same time it can serve as implementation guideline for you.
We didn't discuss this any further and I don't know if that has been done yet, but should I create a wiki page for this (I know we settled on that) for this alread, or wait? If I should, where should I put it/how should I name it, etc?
I'll be getting to work on this again this weekend, and I hope to do a bit more then than I did now.
Tom
On Thu, 31 Mar 2011 22:56:40 +0200 Tom Willemsen tom.willemsen@archlinux.us wrote:
Hey guys,
I didn't get a lot done today, but I did get something done. I've written that django management command to get the available isos, iso's can now be fetched by running `./manage.py sync_isos`
I had some questions as well:
- there will need to be some kind of "are you human test" on the input form (a simple question/response check might do fine, like "which Linux distro is this about? (just the 4-letter word)")
I notices in packages/views.py in the FlagForm class that you are using a textfield to determine you're not getting spambot entires by requiring it remains empty and keeping users from seeing it, would this also be adequate? Seems clever and easy.
maybe, let's just try it and see.
- Can you provide a sample static page called "help" or something? When I have a setup that I can test on, I can start writing the contents for that help page. It will be needed anyway for users who want to use the app, and at the same time it can serve as implementation guideline for you.
We didn't discuss this any further and I don't know if that has been done yet, but should I create a wiki page for this (I know we settled on that) for this alread, or wait? If I should, where should I put it/how should I name it, etc?
yeah make a wiki page, just name it after the name of your program. so far, I think 'RTF' ("releng-testbuild-feedback") is the least crappy name we came up with.
I'll be getting to work on this again this weekend, and I hope to do a bit more then than I did now.
Tom
Hey Tom, what's the status of the app? everything going ok? if you don't have enough time, I could search for more contributors.
Dieter
On 10 Mar 22:01, Dieter Plaetinck wrote:
okay, so no authentication reuse. then i guess the next best thing is to ask a (nick)name, email address and response to a simple "are you human" question. that should work well enough, except it would allow people to purposely add "everything is okay" reports with a bogus name/ email address, which would cause me to mark iso's as "verified enough" and make an official release of crappy isos. (or the inverse: they can mark as everything as broken). if users would be authenticated I think there would be less chance for abuse.
So how would we do that? a question like which arch is best? or should I start looking at captcha or something?
Another idea might be (although might cause other problems of course, and would require more effort on the user's part) that results have to be verified, like forum logins and such. As I think that everyone who takes the time to use a web interface to enter these questions might already be motivated enough to also respond to a simple single-click verification, perhaps. And I don't think many people would test a single iso all that many times, at most they might enter a new result if they changed their settings from last time. But it still might not be the nicest idea.
Until we write something that does this automatically, like we talked about, somewhere after the installation through a cli app, some user involvement in the process of entering this data will be required.
However, if that is not something we want (it would for example imply I need push access to the repo) or have time to setup right now, I can live with the next best thing: maintaining the help page on the wiki. (I don't like the idea of managing the page content through the webinterface, I want version control, so let's keep this webapp simple and let's not poorly reinvent a wiki)
So what is it then you have against the wiki exactly? just the fact that it's managed through a web interface? Since I thought the wiki was version controlled anyway (looking at the history tab sure gives me an idea much like it) and I thought it would be the logical place for any arch-related documentation? I don't want to be a pain I just don't follow completely :)
On 10 Mar 10:04, Dan McGee wrote:
Nice! I'd be glad to have this hosted on the main site (I'm the maintainer). Overall things look sound; sure there are some style/convention/etc. differences but nothing hard to fix up.
- View page: right now it is making 98 queries (one for each of those
max values). Not that this page is probably going to see loads of traffic but we can surely improve upon that.
I would prefer to change that, yes, I was having a lot of trouble figuring out a nice way to get this kind of structure in the page. So if you or anyone else can point me in another direction it would be much appreciated.
- Add page: Using radio buttons for widgets here would probably be
smarter than the dropdowns, otherwise field entry is excruciatingly painful. See the forms documentation for how to change the default widget.
I'll look at it, that does sound like it could be nicer.
- models, Test.ms: totally un-descriptive field name, and you didn't
give it a nice field name. Can't this just be "modules"?
Yes I'll do that.
- models, Test: if you name things boot_type instead of boottype,
you'll get nicer default labels on your view and entry screens. Same with the models themselves- Boottype -> BootType.
Ok I'll do that.
- models, Iso: Likely need more than a date field here. We've double
released, etc. It might just be worth having a CharField as no users are going to be entering these anyway, and also having a boolean "active" field or something to determine what actually shows up on the entry and view pages.
Sure, I didn't know what to put in it (or most of the models for that matter) anyway.
- models, get_success_test/get_failed_test: these can be abstracted
into a common proxy model (http://docs.djangoproject.com/en/1.2/topics/db/models/#proxy-models) superclass, from which all these various ISO options inherit from.
Ah ok, I was thinking of using a base model for that, but in the interest of being done as soon as possible I used this way first. I'll look at this.
- tests.py: yes, we should be writing tests, but at the least, get rid
of the default garbage in this file.
Oh heh, I didn't write that file, I'll remove it.
- admin.py: convention is not to use * imports; I know it works here
and makes things cleaner but I've eradicated the codebase of them otherwise.
Ok I'll get rid of all of them.
- models.py: "# Create your models here." -> not needed
You're right.
- urls.py: You aren't using info_dict anymore; you have a blank second
pattern definition?, and I'd prefer you follow indentation patterns of something like packages/urls.py.
Right again, I'll look at the indentation, I was thinking about that anyway, it's not nice to not adhere to someone else's style rules, I just sent first and wanted to clean up later.
- views.py, add: A lot of extra comments. "Create your views here.",
all the "form been submitted" trailing stuff. For the return, I've switched to using mostly direct_to_template to avoid some of the boilerplate- see the end of flag() in packages/views.py around line 373 for an example.
I'll look at that too.
- views.py:, view: I see some things are _choices, some are _list- why
the disparity?
Hehe, that's because in the beginning I thought it might be better to use the choices parameters for some things, but then later I switched and I apparently didn't want to rename the fields then, I think I was in a rush at the time, sorry.
- isotest/templates: why is this in a different directory from the
standard layout?
Oh I'm sorry, that should have already been removed. That's there because I read in the django documentation that templates _should_ also be looked for in an app's directory, I thought it'd be nicer if I completely seperated my app from the rest of archweb so it could have been placed in any project if it wasn't going to be placed into archweb. But I couldn't get that to work right away so moved on again.
- Finally, I see 4 + 1 templates, but only two views- are these other
ones old, unnecessary, not wired up yet?
They must be from tests with object_list generic functions and such.
Great work! I know I just said a lot but these are mostly minor things, nothing to feel bad about.
Thank you! I appreciate your comments, I will get to work on everything you've mentioned.
Tom
On Thu, Mar 10, 2011 at 11:30 AM, Tom Willemsen tom.willemsen@archlinux.us wrote:
On 10 Mar 10:04, Dan McGee wrote:
- isotest/templates: why is this in a different directory from the
standard layout?
Oh I'm sorry, that should have already been removed. That's there because I read in the django documentation that templates _should_ also be looked for in an app's directory, I thought it'd be nicer if I completely seperated my app from the rest of archweb so it could have been placed in any project if it wasn't going to be placed into archweb. But I couldn't get that to work right away so moved on again.
As a side note, this should have worked in theory- we have the app_directories loader set up in TEMPLATE_LOADERS, since you need this for all the admin stuff, etc.
-Dan
On 10 Mar 12:02, Dan McGee wrote:
On Thu, Mar 10, 2011 at 11:30 AM, Tom Willemsen tom.willemsen@archlinux.us wrote:
On 10 Mar 10:04, Dan McGee wrote:
- isotest/templates: why is this in a different directory from the
standard layout?
Oh I'm sorry, that should have already been removed. That's there because I read in the django documentation that templates _should_ also be looked for in an app's directory, I thought it'd be nicer if I completely seperated my app from the rest of archweb so it could have been placed in any project if it wasn't going to be placed into archweb. But I couldn't get that to work right away so moved on again.
As a side note, this should have worked in theory- we have the app_directories loader set up in TEMPLATE_LOADERS, since you need this for all the admin stuff, etc.
Yes I thought so too, I've tried it with a new project as well, but it just doesn't seem to work.
arch-releng@lists.archlinux.org