Page 1 of 2 1 2 LastLast
Results 1 to 30 of 32
  1. #1
    Join Date
    Nov 2007
    Location
    Paris France
    Posts
    28
    Thanks
    4
    Thanked 33 Times in 6 Posts

    Lurking tool: ConceptArt.org image scrapper! (Edit: Version 2.2!)

    Hi all,
    I'm a bad artist but a long time CA.org lurker. I've just spent a few hours making a tool that will make my lurking easier. I find the message board a bit cluttered and hard to navigate on a small screen, so this page is my solution.
    It displays the most recent images posted on the forum and loads more as you scroll down. I know the tool is a bit slow, it's ok for me but if there's any interest I'll see what I can do to optimize it.

    http://www.antoinegersant.com/misc/lurk/
    Name:  lurk.jpg
Views: 1556
Size:  261.3 KB

    EDIT: Version 2.1 is up and running!

    -- Changelog Version 2.2 --
    · Added tooltips with thread names

    -- Changelog Version 2.1 --
    · Filtering can now be turned ON or OFF (default: OFF) and its defaults setting filters everything out

    -- Changelog Version 2.0 --
    · Much faster
    · Filter pictures by forum
    · Keeps one month worth of pictures
    · Updated every hour
    Last edited by fil_razorback; December 27th, 2012 at 07:22 PM.


  2. Hide this ad by registering as a member
  3. The Following 26 Users Say Thank You to fil_razorback For This Useful Post:

    + Show/Hide list of the thanked


  4. #2
    Join Date
    Dec 2010
    Location
    Capital of TX
    Posts
    796
    Thanks
    247
    Thanked 261 Times in 201 Posts
    This is so neat man! Thanks for sharing and please if you can work on it more!
    Sketchbook Help me improve! Animations
    “To be an animator you have to have a sense of the dramatic, a feel for acting. You have to be a storyteller.”-Marc Davis
    "It's not the size of the dog in the fight, it's the size of the fight in the dog.” ― Mark Twain
    My peeps
    Oghren Care
    Lakai
    Etern

  5. #3
    Join Date
    Mar 2008
    Location
    Falmouth, Cornwall, UK
    Posts
    315
    Thanks
    182
    Thanked 94 Times in 76 Posts
    This is so awesome. Thanks Fil I've already found a few really interesting threads I probably wouldn't have found otherwise.
    My Portfolio

    My Sketchbook
    Please feel free to add a comment, I appreciate all feedback!

  6. #4
    Join Date
    May 2012
    Location
    Buenos Aires, AR
    Posts
    269
    Thanks
    82
    Thanked 98 Times in 74 Posts
    Nice! That's a pretty good tool for discovering new threads. I think it'd be much be better if it was part of ca.org, or at least hosted in these servers. That way ca doesn't have to spend bandwidth on it.

    Btw, you could try caching the first set of images every hour so it loads faster initially.
    Great idea razorback
    "Great job guys! I love you. You're fired."

    Sketchbook! Me vs Anatomy (and other things)

  7. #5
    Join Date
    Jan 2006
    Location
    Brazil
    Posts
    553
    Thanks
    201
    Thanked 80 Times in 66 Posts
    Very cool!
    My website

    Facebook Page

    New CA Sketchbook

    " The scientific and generally accepted-in-art term for this is "You're fuckin' screwed, dude..." " Ilaekae - May 16th, 2009

  8. #6
    Join Date
    Nov 2008
    Location
    North Carolina
    Posts
    794
    Thanks
    673
    Thanked 379 Times in 256 Posts
    That's great! Yeah I would like to see what you could do to optimize it. Sometimes its better just to look at the pictures and not be distracted by all of the text.


  9. #7
    Join Date
    Aug 2004
    Location
    Tennessee
    Posts
    1,306
    Thanks
    219
    Thanked 171 Times in 109 Posts
    I love it. I wonder if it would be possible to make it just for "Finally Finished" or "Sketchbook" sections?

  10. #8
    Join Date
    Nov 2007
    Location
    Paris France
    Posts
    28
    Thanks
    4
    Thanked 33 Times in 6 Posts
    Thanks for your feedback! I'm working on an updated version =)

  11. #9
    Join Date
    Dec 2008
    Location
    SEA, WA
    Posts
    881
    Thanks
    260
    Thanked 194 Times in 160 Posts
    DUDE! Awesome!

    I love it, and will be checking back to see what you can do to optimize it. Good luck, and thank you!
    -I often post from my phone; so please excuse the typos
    Sketchbook

  12. #10
    Join Date
    Jan 2008
    Posts
    474
    Thanks
    120
    Thanked 194 Times in 69 Posts
    This looks like something that could be a major addition to CA. If this was a part of the site (with some more improvements like faster loading time, etc.) I could possibly be more active here. Hmm.

  13. #11
    Join Date
    Jul 2002
    Location
    Austin TX
    Posts
    11,707
    Thanks
    2,091
    Thanked 11,435 Times in 2,935 Posts
    Hey Razor...Thanks for making that. Plz check your PM's when you get a chance. Awesome.


    JM

  14. #12
    Join Date
    Nov 2007
    Location
    Paris France
    Posts
    28
    Thanks
    4
    Thanked 33 Times in 6 Posts
    @Jason Manley > Replied
    @Others > Thanks!

    Version 2 is up and running!
    Changelog:
    - Much faster
    - Filter pictures by forum
    - Keeps one month worth of pictures
    - Updated every hour

    I hope you like it, I certainly had fun making it. Here it is.

  15. The Following 3 Users Say Thank You to fil_razorback For This Useful Post:


  16. #13
    Join Date
    Nov 2011
    Posts
    289
    Thanks
    104
    Thanked 67 Times in 59 Posts
    Very awesome!
    I can also sense some work coming your way, huh ? That's great, man!

    I'm looking forward to seeing updates with this program/site. Very useful!

  17. #14
    Join Date
    Aug 2012
    Location
    Australia
    Posts
    218
    Thanks
    118
    Thanked 31 Times in 29 Posts
    Brilliant work!
    2.0 is definitely faster than the previous version.

    The filtering probably needs a bit of work IMO.

    Example:
    When page is opened, all forums are selected as one checkbox.
    Along with all the other forums/sub-forums available deselected.
    Enabling people to select what they want to see specifically without having to uncheck all the checkboxes.

    Then, for example if this does get included into the site, have it so that upon login it can remember what forums you visit so once the page is opened it goes straight to your "favorites" instead of all selected.

    Just my two-cents.

    Again, great work!
    Ever just woken up and gone "shit, does the world around me exist"?

  18. #15
    Join Date
    Aug 2012
    Location
    Australia
    Posts
    218
    Thanks
    118
    Thanked 31 Times in 29 Posts
    Brilliant work!
    2.0 is definitely faster than the previous version.

    The filtering probably needs a bit of work IMO.

    Example:
    When page is opened, all forums are selected as one checkbox.
    Along with all the other forums/sub-forums available deselected.
    Enabling people to select what they want to see specifically without having to uncheck all the checkboxes.

    Then, for example if this does get included into the site, have it so that upon login it can remember what forums you visit so once the page is opened it goes straight to your "favorites" instead of all selected.

    Just my two-cents.

    Again, great work!
    Ever just woken up and gone "shit, does the world around me exist"?

  19. #16
    Join Date
    Nov 2007
    Location
    Paris France
    Posts
    28
    Thanks
    4
    Thanked 33 Times in 6 Posts
    Thanks for the suggestion, I think it's a lot more convenient now.
    Version 2.1

  20. The Following User Says Thank You to fil_razorback For This Useful Post:


  21. #17
    Join Date
    Aug 2004
    Location
    Tennessee
    Posts
    1,306
    Thanks
    219
    Thanked 171 Times in 109 Posts
    I just noticed the "Filter" option. That's amazing! Thanks for this.

  22. #18
    Join Date
    Feb 2012
    Location
    England
    Posts
    85
    Thanks
    12
    Thanked 24 Times in 17 Posts
    Wow, awesome. This is really handy since you can pick and choose quickly. It would be great if somehow you could see the thread name by hovering over the picture, which could quickly put the picture into context.

  23. #19
    Join Date
    May 2012
    Location
    Buenos Aires, AR
    Posts
    269
    Thanks
    82
    Thanked 98 Times in 74 Posts
    Great update. Feels snappier now
    "Great job guys! I love you. You're fired."

    Sketchbook! Me vs Anatomy (and other things)

  24. #20
    Join Date
    Nov 2007
    Location
    Paris France
    Posts
    28
    Thanks
    4
    Thanked 33 Times in 6 Posts
    Thanks all! Having actual users make me really happy =)
    @Shirobon > Done! It's not *exactly* the thread names (because I haven't been crawling them) but a readable version of their URL. The difference is subtle so I think it will do.

  25. #21
    Join Date
    Dec 2008
    Location
    SEA, WA
    Posts
    881
    Thanks
    260
    Thanked 194 Times in 160 Posts
    I'm speechless! This is truly badass.

    I like the "hover" title aspect. I don't want to see a bunch of title text over all of these images as I am scrolling through; only the ones I hover over. The whole appeal of this site is that it cuts out all the clutter of text and brings you just the images.

    Again, truly awesome.
    -I often post from my phone; so please excuse the typos
    Sketchbook

  26. #22
    Join Date
    Dec 2008
    Location
    SEA, WA
    Posts
    881
    Thanks
    260
    Thanked 194 Times in 160 Posts
    I'm speechless! This is truly badass.

    I like the "hover" title aspect. I don't want to see a bunch of title text over all of these images as I am scrolling through; only the ones I hover over. The whole appeal of this site is that it cuts out all the clutter of text and brings you just the images.

    Again, truly awesome.
    -I often post from my phone; so please excuse the typos
    Sketchbook

  27. #23
    Join Date
    Dec 2008
    Location
    SEA, WA
    Posts
    881
    Thanks
    260
    Thanked 194 Times in 160 Posts
    PS: if there is a way to extend the backlog of images to more than a month, that would be great. I was just lurking the Tutorial section, and it looks pretty skimpy because not a lot has been uploaded in the last month.

    Maybe you can keep the bigger subforums (Sketchbook, IFF, WIP, etc) at one month, but increase the smaller/less visited forums?
    -I often post from my phone; so please excuse the typos
    Sketchbook

  28. #24
    Join Date
    Dec 2008
    Location
    SEA, WA
    Posts
    881
    Thanks
    260
    Thanked 194 Times in 160 Posts
    PS: if there is a way to extend the backlog of images to more than a month, that would be great. I was just lurking the Tutorial section, and it looks pretty skimpy because not a lot has been uploaded in the last month.

    Maybe you can keep the bigger subforums (Sketchbook, IFF, WIP, etc) at one month, but increase the smaller/less visited forums?
    -I often post from my phone; so please excuse the typos
    Sketchbook

  29. #25
    Join Date
    Apr 2012
    Location
    Ireland
    Posts
    122
    Thanks
    99
    Thanked 34 Times in 23 Posts
    Wow! This is awesome!
    The only recommendation I have is maybe you could have a "search-by-keyword" function as well as the filters.

    I know that CA has an image search function but I prefer the nice, clean interface you've created to browse pictures
    Can't wait to see how this progresses!

  30. #26
    Join Date
    Nov 2007
    Location
    Paris France
    Posts
    28
    Thanks
    4
    Thanked 33 Times in 6 Posts
    @Syle > I've adjusted the settings, the tool will now keep pictures for 6 months (that will occupy about 1GB on my server, just with thumbnails). However, I cannot crawl data older than a month (because the activity stream doesn't go this far) so it will fill gradually from now on. While I was at it I also started crawling actual thread names: pictures that are already indexed now display their slug (URL friendly name) on rollover but future ones will have their name exactly as it appears on the forum.
    @The-Idle-Elk > Thanks. I'm not too sure about a search function…It is doable but it feels a bit out of scope and I'm not sure I could improve a lot on the experience you get from the original CA.org there.

  31. The Following User Says Thank You to fil_razorback For This Useful Post:


  32. #27
    Join Date
    Nov 2008
    Location
    North Carolina
    Posts
    794
    Thanks
    673
    Thanked 379 Times in 256 Posts
    Its working great. You've got the speed issue pretty well fixed. Now all I have to do is scan through the thumbnails for something interesting. Before you had to dig through all of the threads and you never knew if you were missing out on something cool. Thanks!


  33. #28
    Join Date
    Mar 2008
    Location
    Michigan
    Posts
    466
    Thanks
    37
    Thanked 101 Times in 86 Posts
    Yeah I'm not surprised Jason hopped on this... this would be really cool to see integrated into the site somehow. There are portions of this forum that I am hardly aware of and I rarely click through more than a few sketchbooks. Very cool!

  34. #29
    Join Date
    May 2010
    Location
    Stockholm/Sweden
    Posts
    884
    Thanks
    577
    Thanked 627 Times in 300 Posts

  35. #30
    Join Date
    Mar 2009
    Location
    Philadelphia, PA
    Posts
    470
    Thanks
    539
    Thanked 395 Times in 174 Posts
    This is fantastic!!
    "Contrary to the belief of the layman, the essential of art is not to imitate nature, but under the guise of imitation to stir up excitement with pure plastic elements: measurements, directions, ornaments, lights, values, colors, substances, divided and organized according to the injunctions of natural laws. While so occupied, the artist never ceases to be subservient to nature, but instead of imitating the incidents in a paltry way, he imitates the laws."-Andre Lhote

    Web, FineArt, Sketchbook

Page 1 of 2 1 2 LastLast

Similar Threads

  1. Painter's Image Hose as a Concept Tool
    By sandman in forum Painter
    Replies: 7
    Last Post: July 24th, 2010, 08:48 PM
  2. conceptart.org image feed?
    By Guy Montag in forum Artist Lounge
    Replies: 2
    Last Post: September 22nd, 2009, 06:49 PM
  3. Best image masking / cut-out tool?
    By TsunamiZ in forum Photoshop
    Replies: 1
    Last Post: March 14th, 2007, 09:46 AM
  4. Art: the scrapper
    By DeR-kLoT in forum Finished Art
    Replies: 5
    Last Post: December 5th, 2005, 07:11 PM
  5. scrapper
    By characteranim78 in forum Art Critique Center
    Replies: 2
    Last Post: July 30th, 2005, 11:12 PM

Members who have read this thread: 0

There are no members to list at the moment.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Designed by The Coldest Water, we build the coldest best water bottles, ice packs and best pillows.