16 replies to this topic

[Josh]

Neolib 2

 

Neolib is a python library which aims to automate the cult classic web based game, Neopets. Neolib automates the game from the ground up by centralizing actions around the Neopets user account. The library includes strong functionality for performing complex tasks as well as built-in querying of a user's assets.

 

Neolib is built upon python's famous requests library for handling HTTP communications and the powerful lxml library for parsing HTML content. These two libraries combine to give Neolib a powerful and fast framework for automation.

 

Neolib is aimed for being deployed on a server or cloud environment. The library does not assume that there will be a graphical interface for interacting with the library. Rather, it assumes the code will either be ran in a script on a server or as part of a grander program with a web interface.

 

Neolib is still in a very early stage of development. As such, things are expected to change, up to and including the base classes. If you intend on using the library in this state please ensure you check back frequently with the master branch for changes.

 

Neolib is also looking for developers interested in contributing to the project. A very detailed and useful primer can be found in the links below. Please read it and understand it fully before making contributions to the project.

 

Documentation: http://neolib2.readt....org/en/latest/

Contribution Primer: http://neolib2.readt...ent/primer.html

 

Author: Josh

Github: https://github.com/jmgilman/neolib2

Documentation: http://neolib2.readt....org/en/latest/

Example Programs: https://github.com/j...an/neolib2_apps

Version: 0.1 (Beta)

 

As most have guessed, this is the second iteration of my Neolib library. Since Neopets is undergoing new management, some things have changed and I've deemed it necessary to simply rewrite the library from scratch rather than attempting to fix the first iteration. It also gives me an opportunity to implement new ideas like core classes and switching over to lxml from BeautifulSoup.

 

This is a great opportunity for new Codexians or non-experienced programmers to contribute towards something and learn a lot about Python. The current iteration of the library includes a lot of concepts from both the functional and object oriented disciplines. If you're interested please read through the primer and contact me to be added on as a contributor. 

 

As the library progresses I will be releasing example programs to help stimulate use of the library. This programs will be publicly available and open source via the same Github repo. 

 

Enjoy!

Edited by Josh, 19 November 2014 - 09:30 PM.

[redlion]

Dope.

[Irradium]

Looks nice. Mind if I ask why you're using lxml instead of BeautifulSoup?

[Josh]

Looks nice. Mind if I ask why you're using lxml instead of BeautifulSoup?

 

It's outdated and I've found cases where it can't handle the horrible stuff the Neopets team calls HTML. Also, benchmark tests have shown lxml to be considerably faster. Finally, xpath is awesome! 

[Dan]

Do you have a roadmap? Are you accepting pull requests?

[Josh]

Do you have a roadmap? Are you accepting pull requests?

 

Absolutely accepting pull requests (that's why I wrote the primer to help get started). 

 

I'm going to start adding in a roadmap on Github tonight. Check back in an hour or so on Github and I should have some milestones/issues created.

[Dan]

Absolutely accepting pull requests (that's why I wrote the primer to help get started). 

 

I'm going to start adding in a roadmap on Github tonight. Check back in an hour or so on Github and I should have some milestones/issues created.

 

Awesome.

[Josh]

Did some major updates today. About 90% of the way to the first stage of beta (only item remaining is main shops). To celebrate I've added an example User Shop Autbuyer script to the repository. Does all the steps of a UAB in less than 25 lines of non-fluff code:

 

usr = User(username, password)
w = Wizard(usr)
 
print('Logging in...')
if not usr.login():
    print('Login failed! Aborting....')
    exit()
 
print('Initiaing...')
while True:
    # Search for the item
    print('Searching...')
    try:
        r = w.search(item, max=max)
    except WizardBanned:
        print('Shop wizard banned. Aborting...')
        exit()
 
    # If we have any results we should buy them all
    if r:
        print('Found ' + str(len(r)) + ' of ' + item)
 
        bought = r.buy()
        if bought:
            print('Successfully purchased ' + str(len(bought)) + ' of ' + item)
 
    # Wait some random time for realism
    time.sleep(random.randint(5, 10))

[Josh]

Updated with support for Main Shops, including an OCR cracker. It now has everything to build a pretty solid MAB.

[Eefi]

Great to see you contributing Your code looks really clean and organized too. ^^

[Zombipster]

Hi, i'm a Mac user and when i try to run the command  "pip install -r requirements.txt" it says to me:

 

Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 262, in run
    for req in parse_requirements(filename, finder=finder, options=options, session=session):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1631, in parse_requirements
    req = InstallRequirement.from_line(line, comes_from, prereleases=getattr(options, "pre", None))
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 172, in from_line
    return cls(req, comes_from, url=url, prereleases=prereleases)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 70, in __init__
    req = pkg_resources.Requirement.parse(req)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/_vendor/pkg_resources.py", line 2667, in parse
    reqs = list(parse_requirements(s))
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/_vendor/pkg_resources.py", line 2605, in parse_requirements
    line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec")
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/_vendor/pkg_resources.py", line 2573, in scan_list
    raise ValueError("Expected "+item_name+" in",line,"at",line[p:])
ValueError: ('Expected version spec in', 'pillow=2.6.1', 'at', '=2.6.1')

Storing debug log for failure in /Users/Luigy/Library/Logs/pip.log

 

Then, I mannualy install the dependencies (requests, lxml and pillow) and it looks fine. I run setup.py in that way: "sudo python setup.py install" and it looks fine. So, when I'm going to python and write "from neolib.user.User import User"

it says:

 

Traceback (most recent call last):
  File "untitled.py", line 1, in <module>
    from neolib.user.User import User
  File "/Library/Python/2.7/site-packages/neolib/user/User.py", line 4, in <module>
    from neolib.http.Page import Page
  File "/Library/Python/2.7/site-packages/neolib/http/Page.py", line 6, in <module>
    from neolib.http.HTMLForm import HTMLForm
  File "/Library/Python/2.7/site-packages/neolib/http/HTMLForm.py", line 1, in <module>
    from collections import UserDict
ImportError: cannot import name UserDict
 

I dont know what to do Surely it is a noob mistake, but I need your help

[Josh]

Hi, i'm a Mac user and when i try to run the command  "pip install -r requirements.txt" it says to me:

 

Then, I mannualy install the dependencies (requests, lxml and pillow) and it looks fine. I run setup.py in that way: "sudo python setup.py install" and it looks fine. So, when I'm going to python and write "from neolib.user.User import User"

it says:

I dont know what to do Surely it is a noob mistake, but I need your help

 

This was written for Python 3, it looks like you're using Python 2.7.

[Zombipster]

that's right. I was on python 2.7. Now i'm on python 3.4.2 and it works fine thank you

[Josh]

V0.1 Beta was released today. With this release comes a PyPi package. The library can now be installed with:

 

$ pip3 install neolib2

 

Or upgraded with:

 

$ pip3 install neolib2 --upgrade

[Josh]

I've started another repository for holding example programs that use this library:

 

https://github.com/j...an/neolib2_apps

[Josh]

Good news! As part of the cloud integration initiative this library now has a base Docker image hosted on the DockerHub. 

 

This means you can now write Dockerfile's and use this image as the base for deploying your neolib based programs to the cloud in a controlled and easy to monitor manner. I'll update this thread again sometime this week with examples on exactly how to do this. 

 

Repo: https://registry.hub...gilman/neolib2/

sudo docker pull jmgilman/neolib2

Edited by Josh, 03 December 2014 - 08:20 AM.

[Josh]

After a hiatus for work, development for this library will resume this week. The focus will be pushing for 1.0 with revamping the way parsing information is stored and continued performance/bug fixes. 

 

An effort to ensure compatibility with Cython will also be a focus to provide the ability to natively compile GUI applications with this library (an alternative to the cloud approach).