* update gitignore

* function stubs for hosted model inference client

* add dill to pyproject.toml

* move to experimental folder

* dump dill to s3 directly, add a test

* update pyproject.toml to be less completely-non-portable

* add making requests to internal hosted inference

* .

* settle on request/response formats

* return ModelBundle instead of string

* black

* black + pylint

* dill -> cloudpickle

* remove /create from endpoint

* start of create_model_endpoint

* remove currently unused model_name/reference id from request

* comment

* some comments

* start create_run_job

* .

* .

* live debugging changes :D, looks like create_model_endpoint works

* test case for _nucleus_ds_to_s3url_list

* model endpoint async job

* temp test script + some code changes to accomodate rest api

* can send tasks through and receive responses (links to pickles) back

* comment

* start on upload_responses_to_nucleus

* test script

* uploading dummy data to nucleus works I think

* Get uploading of actual predictions working

* replace model bundle upload implementation with signed s3 grabbing

* wip refactor old code

* temp commit

* fix HMI endpoint

* wip cleanup code

* remove deprecated file

* some placeholders

* signed_url -> signedUrl in create_model_bundle

* temp test

* dumb bundle mistake

* fill in the really small wrappers

* test lowlevel functions

* partially fix create-model-endpoint

* handle no gpu case on endpoint creation

* some docs

* more docs

* move ModelBundle out into its own file

* readme

* remove all nucleus-core references from ModelEndpoint

* make ModelEndpoint use low level HMI endpoint

* add docs

* stuff for docs

* docs ++

* docs +++

* docs ++++

* type annotations just cause

* shh

* trim intro blurb

* rename HMI -> Deploy in python

* rename pt 2

* rename some low level functions

* respond to comments ++

* add a find-packages file

* add finding model endpoint requirements

* fix tests

* see what the find_packages does

* fix docs formatting

* clean

* add load_model_fn support to create_model_bundle

* comment out some model ci dependencies

* doc changes

* refactor ModelEndpoint -> AsyncModelEndpoint

* more clean

* refactor s3URL -> url

* refactor out some hmi endpoints into constants

* remove temp testing file

* add some diagrams

* fix bug in model_endpoint.predict

* remove extraneous data in response

* one more extraneous data

* translate https to s3

* add upload score to nucleus

* remove ml-worker from nucleus integration smart open kwargs

* rip

* rip pt2

* detect failures in polling

* rip pt3

* parallelize requests to hmi

* add a batch status for fanciness

* include request lifecycle diagram

* delete kinda silly tests

* remove unnecessary requirements

* add the list_model_bundles endpoint (finally)

* add back some dependencies

Co-authored-by: Yi Xu <[email protected]>

* breaking api changes happened, fixing python client

* make endpoint creation not break

The client operates in two modes, external mode (default) and self-hosted mode.

Also add a few extra functions for the missing API calls (e.g. delete bundle/endpoint, put endpoint)

For uploading model bundles in self-hosted mode, there is an upload_bundle_fn that needs to be passed to the client. This allows clients to specify their own model bundle storage backends. An example implementation is

def upload_bundle_fn(serialized_bundle, bundle_url):
    transport_params = {"session": Session(profile_name="ml-worker")}
    with smart_open.open(bundle_url, "wb", transport_params=transport_params) as f:
        f.write(serialized_bundle)
n.b. the code assumes that the REST APIs for the public route and the internal route are nearly identical, e.g. creating a model bundle is always at POST /model_bundle/ for example, and it takes in a JSON with a key url.

* add sync endpoint support

* oops some mistakes

* clarify difference between sync + async endpoints and batch jobs

* can't think of other criteria

* rename service_name -> endpoint_name

* rename predict fns

* send code of the function to the endpoint

* black

* mypy

* isort

* update to use new stuff

* reformat

Python client handles json/serialized inputs + outputs, also add a bundle_location_fn notion.

Added an EndpointRequest class to encapsulate the options for a request to an endpoint, and an EndpointResponse class to encapsulate a response

* documentation++

* edit

* Add initial implementation of create_model_bundle_from_dir

* Use temp directory instead

* Split into root_dir and base_dir for the zip file paths

* Add base_dir to bundle metadata

* Remove base_image from create_model_bundle_from_dir signature since it's already in env_params

* Pass bundle 'type' column

* Add docstring

* Changing type -> packaging_type for bundles

* 🏴

* pylint

* mypy gets confused

* isort

* update cloudpickle; register_pickle_by_value

* update poetry

* black

* pylint

* add predict_fn_or_cls argument; add cluster parameters

* black

* add code bundle

* rm aws_role and results_s3_bucket as params

* fix tests

* black

* clarify docstring; check if model bundle is correct

* use a single bundle var

* black

* disable too-many-branches linter

* black