Site Administration
Hi,
I've been holding this post back for a while now because, well I love OGA, and I just don't want to be on here complaining about it, but I think we've reached a point where something needs to be said.
Still just to be clear, this is NOT a complaint, it's an expression of concern and an offer to help.
I think it should be pretty clear to all that OGA is no longer getting the administrative attention in needs.
Search is broken. It doesn't return results past January of this year. The issue was reported in February, it's now June. A year ago search broke in the same fashion. It took four months to get fixed, and then broke again just a few months later.
The main page hasn't had a 'news' post since the fix for a spam attack was announced last Novemember. That post itself is not peppered with obvious spam posts.
Furthermore, unless there's something going on that hasn't been announced publicly, I think it's pretty clear that development on the site has basically stopped.
I think the admins are doing an admirable job keeping up with montioring posts for legal issues, etc. and fortunately, artists and developers still seem to be finding and using the site. This is a real testament to the strength of the community and the technology already set up for the site.
However, it doesn't take a Nostadamus to see that over the long run, if the site doesn't get day to day administration or future development, it's going to die. It'll either slowly fizzle out as artists and developers turn away from it because they find it's not working, or one day it'll just crash or disappear and no one will know why or how to bring it back.
Again, this is not a complaint, it's an expression of concern.
I have nothing but love and respect for Bart and what he's created here with OGA. But it's clear whoever is currently handling the admin for this site simply doesn't have the time or inclination for it anymore. I am totally ok with that. For Bart personally, if he wants or needs time away from the site, by all means he should take it. He's certainly paid his debt and then some. If it were up to me, he'd be able to retire happily to the Bahamas anytime he wants. But there must be some process whereby adminstration and development responsibilities for the site can be passed on or even just shared or temporarily delegated to some person or persons who do have the time and interest to do it.
I am offering myself up as one such person, but even more importantly, I'd like to get the process and the conversation started. What can we do to ensure that OGA endures past the current difficulties and beyond?
Scott
This site has:
* No apparent staff - Can't tell who staff is except by the blog posts, where we can assume bart runs the place. If he posts in the forum we can only infer from context that he's staff, it doesn't say it anywhere. There is no staff page with activity indicators or descriptions of their roles so we know who best to get ahold of if there's an issue. Not even the staff themsevles know their exact roles.
* No helpdesk /ticket system - Best we can do is post in the cluttered "Feedback" forums, with a poor search function and hope it doesn't take weeks to be resolved.
* No "contact" link anywhere. It should at least be in the footer, if there was one.
You need to show the users that staff exist and are doing things, and they need to be accessable. I was going to post a tileset I've been working on, but I noticed my account still hasn't been approved, and was annoyed that no one could find my website after I put up something new. Knowing how how things run here I got really demotivated to TRY to contact someone, as I still remember waiting weeks to get a forum post approved after finishing my game. I figured I'd just think about not posting anything here anymore, and look for alternatives. Then I stumbled on this post. It gives me a little hope, but not much. capbros was absolutely right when he said artists are going to turn away. It's fine if bart has no time; I sympathize with that as I don't have much time either, but someone needs to be running the place. Not just minding the store, but running it.
@SpiderDave: sorry about the lack of account approval. Your name doesn't show up in the list of users waiting to be approved, so I wasn't even aware of it. I'll try to remedy that.
I agree about not knowing who is a moderator or administrator. Beyond the "Administrator" medal, I can't tell who else is "staff" myself. As for the feedback forum, I don't feel it's all that bad. Posts are always responded to promptly. The issues are not always resolved immediately, but they are addressed. I don't see a helpdesk ticketing system providing any sort of wild improvement, but I'm not against it either. :)
A place to contact admins is a good idea.
--Medicine Storm
@spiderdave:
Oh no!! That's so sad to hear! Glad this thread caught your eye before we lost you! Hope we can get your account straight quickly, your submissions speak for themselves! It'd be awful if we lost you!
a contact form and a page listing the admins would be a great!
https://withthelove.itch.io/
Just to reassure everyone, we have 13 admins and another 3 editors in addition to about 12 people with other degrees of elevated access. Of those, 4 admins have not been online in the past year.
I think that is actually a fairly sized "staff", though in practice it is at the whim of who has free time at any given moment. We do have an admin forum, but it is not used as extensively as it could be and I agree we could always organise and co-ordinate ourselves better.
I will try to monitor the feedback and admin forums more closely, but i am wary about making a public list of staff with direct contact capability. I am also not sure that a ticketing system is appropriate, we probably just need to be more timely in our responses to the existing systems, rather than creating more lists for us to check.
Someone should come up with a new method to approve the users website link, mine for example is not visible and waiting for approval for well over a year now. It seems each time you change your project website in your profile it gets reset back for approval, but nobody ever checks that and so it stays not visible forever.
@Duion: Oh no! That sucks too!
Do any of the admins know how approval for profile url changes are supposed to work?
Is this just a case of no one getting around to it, or is there something missing in the system to where these things don't get flagged for admin attention?
Any luck approving SpiderDave's account?
https://withthelove.itch.io/
Accounts appear to be flagged for pending approval when the account is new. If the account is somehow auto-unapproved due to altering profile settings or something, the account does not ever re-appear on the list of accounts pending approval. p0ss or some higher-level admin than I may be able to see deeper into the database, but as it stands now, there is nothing letting admins know that accounts need REapproval.
I approved SpiderDave's and Duion's account within minutes of their posts.
--Medicine Storm
@MedicineStorm: REapproval mine to please. I updated the descripting of my website link to better explain my game and pixel art and now it's not visable.
I'm playing with a bunch of assets available on OGA and making it all fit together but as I make this art available I'd like everyone to have an easy link to my game project in-case there's a developer looking for a game engine might just stumble upon it. Thanks ;)
http://duskrpg.blogspot.com/
Done. :)
--Medicine Storm
I have had a look into it. The approval queue was experiencing a strange bug.
The problem was the option which let you switch between approved and non-approved users, it was causing an SQL error and returning 0 results. I have disabled the switch and set it to only display unapproved users, and it is now functioning as intended.
I will post in the admin forum with more details about my findings. In the mean time, if you feel you should have been approved a long time ago, please let us know
Most likely a lot of the site users don't read all the forum threads (and this one doesn't have anything too obvious to do with link/account approvals on the face of it). I'd recommend putting a front page post up about this bug and letting them know it's being addressed. Perhaps even create a specific forum thread dedicated to that issue and link the news post to it so that anyone who's been stick in queue limbo can get some attention.
Not a bad idea. In the meantime, I've reviewed, approved, or removed about 1500 accounts. That should account for the vast majority of active, legitimate users. I removed a bunch of obvious bot accounts (never logged in, no activity of any kind, over 2 years old). If your account needs reapproval still, send me a message. If you're a new user, you don't need to message me. Your account will likely be approved by one of the moderators in a day or two. :)
--Medicine Storm
Whats this Mollom thing now that won't let me post to my own thread?
Red warrior needs caffeine badly.
Mollom is the anti spam service. There is no reason it should be preventing you from posting. I was very careful when marking accounts for approval and spam. Let me see if I can figure that one out. What thread were you attempting to post in?
--Medicine Storm
Just trying to post some more junk in my CC0 scraps thread: http://opengameart.org/forumtopic/cc0-scraps
Red warrior needs caffeine badly.
nothing seems out of order. Neither you nor the thread itself is marked as spam or suspicious. test-me was able to post a response, but it required a captcha-style verification. Was it demanding any similar verification or anything? I'm afraid I've exhausted my somewhat limited admin ability for this one. I don't know enough about how the mollom system is set up to determine why it wouldn't allow you to post.
Perhaps the Mollom system is being tweaked at the moment. Anybody else able to shed light on this? :)
--Medicine Storm
Tried again and still getting:
Not seeing any captcha.
Red warrior needs caffeine badly.
You are definitely not blocked in any way. I made an adjustment that should not affect any legit user, but should allow you to post. Try logging out, logging in, reattempt post.
--Medicine Storm
That did the trick. The Mollom privacy policy stuff it gone from beneath the submission form too.
Red warrior needs caffeine badly.
I also got flagged by Mollom, tyring to point out a new asset in the comments section under a collection page. I just directly messaged the currator instead, but still it seems weird to not be able to comment. For reference, here is the comment (plus many close variations on the same theme) that was rejected:
@DezrasDragons, I just fixed that for you, may have to log out and back in again for it to take effect.
@kagerato That is a good point, I have added a blog post to the home page so others can see the issue.
Ooh! "Featured Art" on the main page! I love it.
--Medicine Storm
Yeah the featured art on the main page is really great!
Featured art feature is awesome!!
https://withthelove.itch.io/
Any updates on the search?
The old "experimental/alpha" solr search worked well for me, but the legacy search and what is in place now really don't... just now I tried looking for "item power up" and ended up getting multiple pages of duplicate results (about 5 or 6 different entries), some of which weren't actually relevant...
Ok, I'm here to keep this thread alive!
We're past six months that search has been broken now.
Again, I do certainly appreciate the work that goes into keeping OGA alive, but is there anything any of us can do to help get this issue addressed?
And could we get an update on the main page at least acknowledging the issue so new-comers and folks that have missed this thread have a way of knowing about the issue and the various work-arounds available?
https://withthelove.itch.io/
capbros. Essentially the situation is this. Based on the Patreon funding drive, the community haven't deemed it worthwhile to fund OGA development, so there has been no OGA development.
I have a young family, a more than full time job and many other interests (like a sweet new VR headset). I can't speak for other admins, but i believe Bart is in a similar situation and all of our admins and mods are volunteers with other commitments.
I am happy to do jobs around the place which take a few of hours on the odd weekend I get free, but I can't really dedicate a solid week to something like setting up and configuring a new Solr search instance.
If we as a community can raise enough money to actually pay Bart's bills, he has graciously said he would end his current contract and work on the site full time. That has not happened, so development has stagnated. As I understand it the current rate of funding and advertisements doesn't actually cover the hosting costs, so Bart is essentially paying for this site to stay up out of the goodness of his heart.
So we are at an impasse, either we find a new trusted volunteer admin with heaps of free time who has extensive experience doing PHP development and is familiar with Drupal 7 and Apache Solr, or we stump up the funds to pay for it. Alternatively we take the path of least resistance and keep doing what we're doing, break fixes and minor improvements.
The current rate of funding does not cover the hosting costs? How is that possible? The current rate of funding could easily cover the hosting costs 5-10 times.
This site gets a lot of traffic and runs some pretty complex scripts. It may require some fancier-than-average hosting features. I don't see how it would cost that much still, but maybe migrating to a new host could cut costs. I have no idea what it would take to do that, though. If anyone here does, is it worth exploring that option?
--Medicine Storm
>Based on the Patreon funding drive, the community haven't deemed it worthwhile to fund OGA >development, so there has been no OGA development.
I think this is a vicious cycle. I personally see no updates for a while on the Patreon page, so I wonder why I should donate more. It seems like giving money away for nothing new. I understand as someone who had sites in the past that there are expenses, but I can't see backend costs. A typical user here would understand even less.
I also wonder how much the hosting is. If high traffic is really a problem, then why do we pay for lots of people to get stuff for free and expect a small number of people to pay for it?
I know I am more coder art than real artist, but a few favorites doesn't really equate to making people want to submit much., It can be hard to repay them for their effort and time also. And it could be more a problem for the real artists, who are the lifeblood of this site.
Feel free to private message me or email me about donations, if you want. I'd rather that stuff not be public. Or if there's a private forum with only friends of the site on it, please add me to that.
Let me play a bit whistleblower, I think hosting is somewhere around 100 dollar per month, at least that is what he told me a while ago. So costs are well covered.
For administration I think he will at least do security updates, otherwise the site would likely already be broken now. On the other hand programmers or system admins are pretty expensive in real life, so the donations are justified.
But you also have to consider that this site already is the by far top receiver of donations in the whole scene. Nobiax for example one of the top donaters overall, is just at 48 dollar per month and he actually produces content. So for the amount of donations for this site you would expect some updates from time to time.
Is the Patreon tracker really still accurate/updated? It never really seems to move, whether up or down. I guess that's possible if it's the same group of loyalists every month.
From a funding and development perspective, the site may be stuck in a tough place. More funding is needed to do both serious curation and administrative work, but the user base is unlikely to pay for it when the status quo works for them. There's no critical point that would force action until the site stops functioning or people stop contributing content. Furthermore, a big part of the problem is that individuals who can afford to contribute serious money would much rather spend it on private contractors, employees, and licenses which they can leverage more easily as revenue generators in the future.
yuck. Let's not get lost in a game of 'where does all the patreon money go?'
I think it's sufficient to leave it at the funding is enough to keep the site up and running but not enough to pay for any active web development on it.
I am actually ok with the site going forward without active feature development. We can all dream of a million ways the site could be improved or new fizwidgets that could be added, but the reality is, the site as it is terrific. Maybe not perfect, maybe not up to all the latest fashions in web programming, but it certainly works well enough, otherwise it wouldn't see the traffic and continuous stream of submissions that it does.
However, I am very concerned that search is broken and has been broken for such a long time. There are a lot of things that can break and stay broken with the site, but search is not one of them.
I am not familar with PHP or Drupal, but I am a pretty seasoned developer and a quick study and more than happy to volunteer my time towards finding and fixing the search problem. Admins may private message me if they feel I could be of help in this capacity.
If there is anyone else from the community who has experience with PHP and Drupal and would like to volunteer their time towards fixing the search issue it'd be great to hear from them.
In the short term, I think it would very, very nice to post something to main page about the search issue.
Something along the lines of:
Search Issues
There is currently an issue with the search indexing on OGA which is preventing works newer than February 2016 from appearing in the search results. The problem is being investigated, however there is currently no ETA for getting it fixed. As a workaround, a 'Title' search option has been added to the search pages. This searches submissions by title only but side steps the indexing issue which is thwarting the main search feature and will return results newer than February 2016.
https://withthelove.itch.io/
If the software was open source, they could put it to github and people could submit fixes. This only leaves code review to the admins.
But I guess it is not possible for some reason
The software the site uses is open source. It would not help much to put it on github, since you would need the database and the files and then you need to setup a webserver to be able to work with it.
Yeah I should clarify, I don't actually know the exact hosting costs and I don't feel comfortable speaking on Bart's behalf about money. I absolutely appreciate how amazing it is that the site is getting donations, and I am personally very grateful to each and every person who helps out with their hard earned dollarydoos.
That said, it is worth acknowledging that as amazing as it is, the current monthly funding rate is less than the daily rate of pay for a Drupal developer in Australia, and less than my rent for a week, I imagine its similar where Bart lives. It may be worth investigating hiring a developer from elsewhere in the world, but given Bart wrote the code for the entire site, he does seem like a logic choice to work on it.
In terms of server load, there are something like 27,000 approved users with profiles and collections and favourites, which is similar to a medium sized forum. The difference is that there are something like 30,000 art entires, each of which can have dozens of files which can each be hudreds of megabytes. The closest analogy would be a media sharing site like a small version of deviantart, soundcloud or youtube. The server load for those kinds of sites typically requires a lot more ads, corporate backing or paid premium accounts. Drupal scales well, but any database can get a bit unwieldy when you regularly do complex queries against multiple tables with millions of cells.
As for open sourcing the code, I know that is the long term intention and one of the patreon funding goals, There is work involved, there are custom modules which need a bit of documentation, unit tests and clean up, and I think the new theme was going to be finished first. There will also be some overhead in setting up and managing the dev community along with testing and accepting new code. It is not a huge amount of work, but it is probably at least a weeks worth. Even then, that isn't going to solve everything, Drupal doesn't handle syncing database commits very gracefully, there is a lot of webserver and app config work that will still have to be manual.
If you're interested in the funding goals and motivations, I recomend reading through Bart's topic
http://opengameart.org/content/support-opengameartorg-on-patreon
The database size is no problem, what counts more is bandwidth and how many people access it.
Bart did not write the code for the entire site, drupal is a framework you chose modules from and then eventually code some modules by yourself if the existing modules are not enough, here are the modules developed by him for opengameart: https://www.drupal.org/user/401025 and he eventually has some scripts that are unpublished.
Most problems would probably be fixed by upgrading or downgrading the modules. The problem with that is, that there is already a big database and files and some modules are not that easy to change, since there is a lot of database entries attached to it. This leads to some issues you see now, like the search not working or views not counting on forum topics, since the counter broke at some time when the module was updated or not fixed when it was broken.
I have no idea how to fix it, my only idea is to reset the things that are broken and start again with a stable module and a clean database.
Just my two cents, I don't think the fault with search lies in the code. Search was working for a long time and then it stopped working. Since there isn't much active development going on with the site, it doesn't seem likely that a code change was responsible. Instead it seems more likely that something elsewhere in the system broke down, eg. a stuck query, something in scheduler not running anymore, max table size exceeded somewhere. Those are just total stabs in the dark, but you get the idea. It's possible the ultimate fix will be a code change, but I'm not sure just spelunking around the code base looking for errors is going to be that helpful. I think it will take a broader view of the entire system and its parts and how they fit together and what could go wrong with those connections to find the problem.
Personally, I would start by looking at search and queires it generates. I'd try running a few manually to see if that gets the same result as the web code displays. Then I'd take a look at the database tables the queries hit and try to understand how they are populated and why they might not be getting the right data anymore.
But again, I don't know anything about Drupal or how OGA is setup so that's just my naive guess at what would be a sensible approach.
https://withthelove.itch.io/
@Duion You are correct, Bart didn't write all of Drupal, but he did write all of the *unique* code on this site. His drupal profile lists his public commits on those projects, but it is nowhere near all the code he has put into this site. I can see at least 5 entirely new and unique modules in the OGA module list, in addition to an entire unfinished project to update and rationalise the entire codebase. That is a lot of custom code for a new dev to learn.
The database size absolutely does matter, since it effects every non-cached dynamic request, of which there are a lot on this site. If you hit a Drupal server with a large number of concerrent non-cached requests against a large number of tables, you'll know about it in terms of page load times. It also makes the site more unwieldy to work with, because the larger your database, the longer Drush commands take to execute, so that even simple tasks can take hours waiting for a script to finish. In enterprise scenarios like on Acquia Cloud Site Forge, we deal with it using a massive front end Akamai cache to try and reduce the number of unique requests as far as possible.
The caching problem gets worse the more unique pages you make it possible for visitors to access, and the more tables each of those pages is calling on. The classic Drupal database chunkers are large views with lots of exposed filters, like the OGA search page. Just do the math on how many pieces of art there are, and then how many possible combinations of filters there are on the art search page. We'd have to be caching something like 3 x 1034 possible URLs, before we even add a search term.
Every user also has multiple pages which are each composed of yet more dynamic views, and we allow end users to create and populate an arbitrary number of new nodes and files. It really is a lot of possible ways for end users to directly utilise CPU time. None of these have any cost or disinsentive for the user, but they do have a cost on the server resources. if people are just downloading art and chatting on the forum that is generally fine, but if they start to do any actually intensive stuff, it can get to be a lot of concurrent intensive CPU jobs, which slow down everything for everyone,
Don't get me wrong, the bandwidth is also an issue, but that is easier to solve with small amounts of money than a CPU bottleneck, because of process locking.
Do we think search is a cpu issue, or were you two just talking generally?
No hope of getting a post to the main page explaining the search issue?
https://withthelove.itch.io/
I used to be a novice to intermediate web designer myself (Javascript, CSS, HTML/DHTML, and a tad bit of XHTML, though I didn't get into php or asp because the web host I used didn't allow it.) Is there anything I can do to help, by chance?
Just note that my schedule is a bit limited and unpredictable because I do work as a caregiver, but I can see what I can do on the side.
I'm glad to see admin is still working on the site, I hope you can soon get rid of that useless spam filter that blocks me all the time while let spambots flood the site with posts.
Here to keep this thread alive, and as it happens, we've recently had two related posts pop up.
One another user has wasted time and energy reporting search issues:
http://opengameart.org/forumtopic/search-broken
Can we really not get anything about this issue posted to the main page? I really don't understand the reluctance to do this. Maybe I am missing something but it only seems natural to communicate an issue of this magnitude to the community.
Two, a report of another issue with the site:
http://opengameart.org/forumtopic/uploading-other-authors-works
I think the workaround here is fair enough for short term use, but it's not hard to see that if used for a long period, it will just lead to more submissions where the submitted (and searchable) information doesn't match the true info. Instead the actual submission info can only be gleaned by a careful reading of the description field. Too many submissions like this threaten the integrity of the site overall. So again, this is an ok fix for the short term, but troubling if left in place too long.
Well again, I want to repeat, I'm writing only out of love. I know the work that goes into keeping this site going and I don't want to be denigarating the effort anyone is or has put into OGA. However, I can't lie, I feel like we are witnessing the fisrt steps of the kind of slow death I described in my opening post to this thread.
Finally, is there anything I can do to help?
https://withthelove.itch.io/
I am lower on the totem pole of admins, so I don't have much sway on the coding and back-end, but my own weakness is understanding drupal and apache solr. If you know about those or can help me track down resources for understanding those, it could put both of us in a better position to resolve the search issue.
--Medicine Storm
Perhaps these may be of use, Medicine?
Drupal Code Documentation Site https://www.drupal.org/documentation
Apache SOLR Resources site: http://lucene.apache.org/solr/resources.html
Search is broken. It doesn't return results past January of this year. The issue was reported in February, it's now June. A year ago search broke in the same fashion. It took four months to get fixed, and then broke again just a few months later.
This seems to be fixed i tried an object and i could find it with tags and title but not with the normal search there seems to be an general problem with the search.
@Rainbow Design: If you sift through the thread here you'll find where pOss added the title search as a work around for the normal search. Not sure why tags would also work, I don't think that was part of his fix but maybe they are also not affected by whatever ails the main search.
https://withthelove.itch.io/
Actually i think one of the mayor problems is that any search returns results when the site has any of the terms so if you use is or the in the search you will automatic get 1000 pages.
I wonder if he might change this to so all items need to be in the result.
Still keeping the faith here, are we really going to pass the 1 year mark without even an announcement about the search issues?
I also just noticed that the chat logs stop at October 8, 2015. I guess it's been a bit since I joined the chat, but I usually like the look through the recent log before logging on just to see if I am jumping into the middle of discussion or there are any recent questions/comments I can address.
https://withthelove.itch.io/
Pages