Quantcast

Jump to content


Photo

Using Amazon EC2


  • Please log in to reply
224 replies to this topic

#101 Cript

Cript
  • 1,939 posts


Users Awards

Posted 07 July 2013 - 10:26 AM

Every modern application you run on any machine will generally multi-threaded. That's how applications can run something in the background and still allow user input (e.g. allowing you to click the stop button in Abrosia even though it's currently refreshing a shop page).

 

 

Your point about keep-alives is true, but what aspect of multi-threading aids that scenario? Having multiple threads doesn't get around the inability to keep a connection open between requests

 

 

I don't really see what you're getting out - you're a bit mistaken here:

  • Every application is threaded.
  • Nothing stops you sending multiple buy requests in one stock with one thread (this is generally why we use asynchronous requests for HTTP)
  • Assuming you mean using threads to send requests, you still won't "avoid" any lag on neopets servers because that's out of any application's control

Using multi-threading as a way of using multiple accounts to monitor shop restocks or buy on side shell accounts can be beneficial, but you're all way off of the mark with your reasoning behind why.

 

It's important to remember that just applying the word 'THREADING' or 'MULTI-THREADED' to an application is just a buzzword that people regard as a synonym for 'fast' or 'efficient' - but that isn't the case - you have to actually know what you're doing and have a decent understanding of thread management for it to offer any kind of improvement.

 

It's also a pretty quick and easy way to make your application monstrously complex and difficult to support.

 

A typical program would function like this:

 

Send SYN to Neo. Wait for SYN, ACK to come back. Send ACK. Now, send HTTP Request to view page. Parse response for item you want. If matched, send SYN to Neo. Wait for SYN, ACK to come back. Send ACK. Now, send POST to buy item. Multi-threading, while not eliminating this inefficiency, minimizes its effect because you'll be sending X amount of parallel requests and POSTs...The more latency involved, the more jitter can effect you. Multi-threading basically provides for the possibility that one of the X threads will be faster than an average single-thread which is huge in Attic where there's only quantity-1 of a great item.


As I'm sure you've noticed, Neo's environment is typically shit. You can make a request, sit for 30 secs waiting for a response, but then re-click your object and get an instant response. There's a fair chance that 1 in X threads will have a good response which again, defeats a random person subject to Neo's crap.



#102 Dan

Dan
  • Resident Know-It-All

  • 6,321 posts


Users Awards

Posted 07 July 2013 - 11:07 AM

Technical information on a specific AA'er can come from whoever produces it and releases it. I haven't nor have I ever been in charge of that part. All I did was have it done. I'm fairly certain the new update in Valorous' suite uses multi-threading. Go pester him for information about it.


I'll repeat myself - I don't care about getting information from you or anybody else - my main concern is that you probably shouldn't use words you don't know the meaning of mate!

#103 ShadowLink64

ShadowLink64
  • 16,713 posts


Users Awards

Posted 07 July 2013 - 05:46 PM

A typical program would function like this:
 
Send SYN to Neo. Wait for SYN, ACK to come back. Send ACK. Now, send HTTP Request to view page. Parse response for item you want. If matched, send SYN to Neo. Wait for SYN, ACK to come back. Send ACK. Now, send POST to buy item. Multi-threading, while not eliminating this inefficiency, minimizes its effect because you'll be sending X amount of parallel requests and POSTs...The more latency involved, the more jitter can effect you. Multi-threading basically provides for the possibility that one of the X threads will be faster than an average single-thread which is huge in Attic where there's only quantity-1 of a great item.

As I'm sure you've noticed, Neo's environment is typically shit. You can make a request, sit for 30 secs waiting for a response, but then re-click your object and get an instant response. There's a fair chance that 1 in X threads will have a good response which again, defeats a random person subject to Neo's crap.


I agree with Cript that this was the issue we were seeing with sending single requests, and is why opening many synchronous connections ends up getting you more in the end. Cript and I identified this problem with Abrosia a long time ago.
 

You can also stop using Abrosia since the code is a mess. You can risk threading, but I think I might have killed that technique.

Thanks for telling everyone to stop using Abrosia. It's this know-it-all attitude of yours that made sure I told you nothing more about what I was up to, especially when you would PM me probing for information.

Abrosia does have the ability to launch synchronous requests to the AA, but I did not make it a public feature. That would invite all sorts of chaos in the Attic. Your claim that you were getting 80% of the items in the Attic definitely didn't hold true while Abrosia was being ran by Cript and I. I'm assuming when Neoquest created a similar kind of program, things started to even out, but we had some fun watching your complaining in the status updates when you weren't getting anything.

And well, it seems Valorous has included it in his AA release today. I might have to follow suit and make this feature public to keep Abrosia relevant.

#104 ceterisparibus

ceterisparibus
  • 485 posts


Users Awards

Posted 07 July 2013 - 06:10 PM

So if I'm reading it right.....a combination of EC2 and right account names allows you to win it big? But doesn't advanced settings allow you to restock using milliseconds? Unless I'm reading it wrong...



#105 shrouded

shrouded
  • lil'cluck

  • 1,250 posts


Users Awards

Posted 07 July 2013 - 06:20 PM

I agree with Cript that this was the issue we were seeing with sending single requests, and is why opening many synchronous connections ends up getting you more in the end. Cript and I identified this problem with Abrosia a long time ago.
 
Thanks for telling everyone to stop using Abrosia. It's this know-it-all attitude of yours that made sure I told you nothing more about what I was up to, especially when you would PM me probing for information.

Abrosia does have the ability to launch synchronous requests to the AA, but I did not make it a public feature. That would invite all sorts of chaos in the Attic. Your claim that you were getting 80% of the items in the Attic definitely didn't hold true while Abrosia was being ran by Cript and I. I'm assuming when Neoquest created a similar kind of program, things started to even out, but we had some fun watching your complaining in the status updates when you weren't getting anything.

And well, it seems Valorous has included it in his AA release today. I might have to follow suit and make this feature public to keep Abrosia relevant.

 

Cript started after me. He'd PM me for information once upon a time.



#106 Adam

Adam
  • Coffee God


  • 4,356 posts


Users Awards

Posted 07 July 2013 - 06:21 PM

So if I'm reading it right.....a combination of EC2 and right account names allows you to win it big? But doesn't advanced settings allow you to restock using milliseconds? Unless I'm reading it wrong...

k32adv.png

 

Yeep - Using my shells, I get buy times from as low as 134ms - 350ms. Those are the ranges I've seen from doing this for the last ~6 months.


Edited by Adam, 07 July 2013 - 06:23 PM.


#107 shrouded

shrouded
  • lil'cluck

  • 1,250 posts


Users Awards

Posted 07 July 2013 - 06:22 PM

k32adv.png

 

Yeep

 

You can lower the attic page to a millisecond amount as well. Some of them go to 0 as well, but I can't particularly remember which since it's been a very long time.



#108 Cript

Cript
  • 1,939 posts


Users Awards

Posted 07 July 2013 - 06:24 PM

Cript started after me. He'd PM me for information once upon a time.

 

And I recall you not sharing any info at all...then when SL and I figured everything out on our own, got to the point where we couldn't miss, and then you decided to stop and start sharing with everyone else. :)



#109 Adam

Adam
  • Coffee God


  • 4,356 posts


Users Awards

Posted 07 July 2013 - 06:24 PM

You can lower the attic page to a millisecond amount as well. Some of them go to 0 as well, but I can't particularly remember which since it's been a very long time.

These seem to be working well for me. I feel like these super fast buy times are crazy enough lol.



#110 Cript

Cript
  • 1,939 posts


Users Awards

Posted 07 July 2013 - 06:26 PM

So if I'm reading it right.....a combination of EC2 and right account names allows you to win it big? But doesn't advanced settings allow you to restock using milliseconds? Unless I'm reading it wrong...

 

Abrosia allows you to set 0.001s as a wait time but that's not the source of most of the delay. You're looking at 3 round trips to setup the tcp connection, 1 round trip for the GET /attic page and then a bunch of round trips to receive the page contents + ACK each packet. EC2 allows you to minimize the round trip time (latency) while certain account attributes & the most efficient AB possible allows you to minimize client-side load time.



#111 shrouded

shrouded
  • lil'cluck

  • 1,250 posts


Users Awards

Posted 07 July 2013 - 06:26 PM

And I recall you not sharing any info at all...then when SL and I figured everything out on our own, got to the point where we couldn't miss, and then you decided to stop and start sharing with everyone else. :)

 

Or when Alex could no longer help me and I didn't want to restart everything from scratch. Not to bring him into this, but after that point it was just going to be too much work with school. It was fun competing with Laura, but that was 1.5 years ago. I didn't share with anyone. I mentioned EC2 in a status update, and then 2 months ago when I went into TC people were using it.



#112 ceterisparibus

ceterisparibus
  • 485 posts


Users Awards

Posted 07 July 2013 - 06:44 PM

k32adv.png

 

Yeep - Using my shells, I get buy times from as low as 134ms - 350ms. Those are the ranges I've seen from doing this for the last ~6 months.

 

Abrosia allows you to set 0.001s as a wait time but that's not the source of most of the delay. You're looking at 3 round trips to setup the tcp connection, 1 round trip for the GET /attic page and then a bunch of round trips to receive the page contents + ACK each packet. EC2 allows you to minimize the round trip time (latency) while certain account attributes & the most efficient AB possible allows you to minimize client-side load time.

 

Ah I see. So it's more of a ping/latency issue then? In that case, I suppose that EC2 greatly alleviates the time required, but abrosia's advanced settings are also a good means of fast buytimes in the attic. Am I reading it right?

 

....wait there's account attributes as well?? wow thats complicated



#113 ShadowLink64

ShadowLink64
  • 16,713 posts


Users Awards

Posted 07 July 2013 - 10:01 PM

Ah I see. So it's more of a ping/latency issue then? In that case, I suppose that EC2 greatly alleviates the time required, but abrosia's advanced settings are also a good means of fast buytimes in the attic. Am I reading it right?
 
....wait there's account attributes as well?? wow thats complicated

Yes, the slowest part of the process becomes the communication between your computer and Neopets' servers. EC2 gives you lower latency, which is why you can get items faster than anyone on a typical residential connection.

#114 shrouded

shrouded
  • lil'cluck

  • 1,250 posts


Users Awards

Posted 07 July 2013 - 10:11 PM

Yes, the slowest part of the process becomes the communication between your computer and Neopets' servers. EC2 gives you lower latency, which is why you can get items faster than anyone on a typical residential connection.

 

Oddest part is that between N.Virginia and Oregan I always found N.Va to work better. Although I think it might have been you who told me that, I can't remember.



#115 Futurama

Futurama
  • The Teacher.


  • 3,752 posts


Users Awards

Posted 07 July 2013 - 10:17 PM

Oddest part is that between N.Virginia and Oregan I always found N.Va to work better. Although I think it might have been you who told me that, I can't remember.

 

 

 

 

I like to live dangerously...I go from Japan.



#116 ShadowLink64

ShadowLink64
  • 16,713 posts


Users Awards

Posted 07 July 2013 - 10:18 PM

Oddest part is that between N.Virginia and Oregan I always found N.Va to work better. Although I think it might have been you who told me that, I can't remember.

It's not odd, because Neopets is hosted in New York City. Neopets hasn't been hosted in California for a while now. And yes, it was me who told you that.

#117 Dan

Dan
  • Resident Know-It-All

  • 6,321 posts


Users Awards

Posted 07 July 2013 - 11:46 PM

I agree with Cript that this was the issue we were seeing with sending single requests, and is why opening many synchronous connections ends up getting you more in the end. Cript and I identified this problem with Abrosia a long time ago.
 
Thanks for telling everyone to stop using Abrosia. It's this know-it-all attitude of yours that made sure I told you nothing more about what I was up to, especially when you would PM me probing for information.

Abrosia does have the ability to launch synchronous requests to the AA, but I did not make it a public feature. That would invite all sorts of chaos in the Attic. Your claim that you were getting 80% of the items in the Attic definitely didn't hold true while Abrosia was being ran by Cript and I. I'm assuming when Neoquest created a similar kind of program, things started to even out, but we had some fun watching your complaining in the status updates when you weren't getting anything.

And well, it seems Valorous has included it in his AA release today. I might have to follow suit and make this feature public to keep Abrosia relevant.

 

 

A typical program would function like this:

 

Send SYN to Neo. Wait for SYN, ACK to come back. Send ACK. Now, send HTTP Request to view page. Parse response for item you want. If matched, send SYN to Neo. Wait for SYN, ACK to come back. Send ACK. Now, send POST to buy item. Multi-threading, while not eliminating this inefficiency, minimizes its effect because you'll be sending X amount of parallel requests and POSTs...The more latency involved, the more jitter can effect you. Multi-threading basically provides for the possibility that one of the X threads will be faster than an average single-thread which is huge in Attic where there's only quantity-1 of a great item.


As I'm sure you've noticed, Neo's environment is typically shit. You can make a request, sit for 30 secs waiting for a response, but then re-click your object and get an instant response. There's a fair chance that 1 in X threads will have a good response which again, defeats a random person subject to Neo's crap.

 

That's really interesting actually. Sounds like it could be fairly dangerous depending on the sheer number of requests you're making though - but either way, still a pretty interesting problem to solve.



#118 shrouded

shrouded
  • lil'cluck

  • 1,250 posts


Users Awards

Posted 08 July 2013 - 02:29 AM

TnT has general anti-bot behavior going on behind the scene. Freaky ran into it with an NP gen in 2010(?), so they'll get you if you're completely over the top.

#119 Lv5Charmander

Lv5Charmander
  • 54 posts

Posted 09 July 2013 - 03:02 AM

Step 5 - Get Windows password and download remote desktop file.
^ I'm stuck at this part, how do you get the password and download the remote desktop file?



#120 Strategist

Strategist
  • Sadmin


  • 9,502 posts


Users Awards

Posted 09 July 2013 - 03:04 AM

Step 5 - Get Windows password and download remote desktop file.
^ I'm stuck at this part, how do you get the password and download the remote desktop file?

Right click on the instance, click 'get windows password', insert your key pair, decrypt password, copy info and open remote desktop from Start Menu > Accessories, enter info in the correct fields and voila! There you have it :D



#121 Lv5Charmander

Lv5Charmander
  • 54 posts

Posted 09 July 2013 - 03:11 AM

Edit: trying to solve it lol
Edit edit: Solved it.


Edited by tututrain, 09 July 2013 - 03:22 AM.


#122 shrouded

shrouded
  • lil'cluck

  • 1,250 posts


Users Awards

Posted 09 July 2013 - 03:34 AM

Right click on the instance, click 'get windows password', insert your key pair, decrypt password, copy info and open remote desktop from Start Menu > Accessories, enter info in the correct fields and voila! There you have it :D


Ty strat :3

#123 Lv5Charmander

Lv5Charmander
  • 54 posts

Posted 09 July 2013 - 04:14 AM

Solved


Edited by tututrain, 09 July 2013 - 04:24 AM.


#124 Strategist

Strategist
  • Sadmin


  • 9,502 posts


Users Awards

Posted 09 July 2013 - 04:18 AM

Have you made sure the instance is actually up and running yet? It takes ~15-20 mins to enable it ;)



#125 Lv5Charmander

Lv5Charmander
  • 54 posts

Posted 09 July 2013 - 04:26 AM

Yea it's up and running, lol. I needed to add new rules to allow connection to it. :/




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users