Neocodex - Neopets Fan Community and Cheats: [ItemDB] Outlier Detection Being Improved - Neocodex - Neopets Fan Community and Cheats

Jump to content

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

[ItemDB] Outlier Detection Being Improved

#1 User is online   ShadowLink64 

  • Group: Administrator
  • Posts: 20,934
  • Joined: 20-March 04

Posted 22 December 2009 - 11:42 PM

Hey all,

As you know, we're continually striving to improve the accuracy of our items database. An issue that has come up recently is some random spikes in the price of an item. Good examples of items that jumped around enough to cause problems were:

Boiled Egg
Visor
and other Tiki Tack items...

Their price would basically multiply many fold due to a couple bad outlier prices being submitted to our database.

With our past outlier detection system, if more than one outlier entered our system, we could easily identify and remove it. However, during additional tests using two outliers in a data set, our outlier detection failed to notice that outliers even appeared in the data. We have now created a secondary system to deal with this.

On another note, the other day, we hit a record of 94 users concurrently submitting data to the database in a one hour period. Good job, automation rocks!

We'll continue to monitor the integrity of our prices and improve the system further if we need to. Thanks everyone for supporting the database thus far. :p
Spoiler
1

#2 User is offline   Icey Defeat 

  • Group: Advanced Member
  • Posts: 11,830
  • Joined: 31-May 04
  • Trader Score:9 (100%)

Posted 23 December 2009 - 01:00 AM

So this is what you guys were going on about in irc. That is kind of a funny problem. Glad to see your on top of things :)
Posted Image
Thanks Jillian <3
0

#3 User is offline   vendetta.inc 

  • Group: Member
  • Posts: 195
  • Joined: 23-September 05
  • Trader Score:3 (75%)

Posted 23 December 2009 - 01:12 PM

First the APer fix, now this. Great work :)
0

#4 User is offline   Andy 

  • Group: Member
  • Posts: 289
  • Joined: 25-October 09

Posted 28 January 2010 - 01:12 AM

TNT broken the outlier detection :(
By retiring a stamp

0

#5 User is offline   Waser Lave 

  • naming monkey
  • Group: Programmer
  • Posts: 17,102
  • Joined: 06-September 04

Posted 28 January 2010 - 04:38 AM

View PostAndy, on 28 January 2010 - 01:12 AM, said:

TNT broken the outlier detection :(
By retiring a stamp


Don't worry, the price will catch up over the next day or so when it has enough information. :p
Posted Image
0

#6 User is online   ShadowLink64 

  • Group: Administrator
  • Posts: 20,934
  • Joined: 20-March 04

Posted 07 February 2010 - 12:04 AM

View PostWaser Lave, on 28 January 2010 - 04:38 AM, said:

Don't worry, the price will catch up over the next day or so when it has enough information. :p

Boy did that price shoot the hell up. :p

Total prices submitted to our database: 15,186,847 and climbing.
Live data size: 27.6 MiB
Historical data size: 203.9 MiB
Good job guys... we have such a great DB and it will only get better this summer. :p
Spoiler
0

#7 User is offline   Waser Lave 

  • naming monkey
  • Group: Programmer
  • Posts: 17,102
  • Joined: 06-September 04

Posted 07 February 2010 - 04:38 AM

View PostShadowLink64, on 07 February 2010 - 12:04 AM, said:

Boy did that price shoot the hell up. :p

Total prices submitted to our database: 15,186,847 and climbing.
Live data size: 27.6 MiB
Historical data size: 203.9 MiB
Good job guys... we have such a great DB and it will only get better this summer. :p


Not too shabby. :p

We still do need to fix that bug where buyable prices just won't go unbuyable, they tend to get to around 98-99k and just rot there even if they are actually unbuyable :/ That can be fixed one of two different ways though so we can look at that when you're back from your larking about at uni. :p
Posted Image
0

#8 User is offline   papersnake 

  • Group: Member
  • Posts: 18
  • Joined: 01-February 10

Posted 12 February 2010 - 05:03 PM

nice to see hardworkers! :D
0

Share this topic:


Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users