Giter Site home page Giter Site logo

azure / autorest.python Goto Github PK

View Code? Open in Web Editor NEW
76.0 289.0 57.0 57.79 MB

Extension for AutoRest (https://github.com/Azure/autorest) that generates Python code

License: MIT License

JavaScript 0.01% Python 99.57% Jinja 0.30% TypeScript 0.12% PowerShell 0.01%

autorest.python's Issues

Introduce mypy in the tests

This issue is about introducing mypy as part of the tox run, for both vanilla and azure. This means:

  • tox should install mypy as a dev dependency
  • tox should run mypy before doing the unittests

mypy should be run on both the generated code and the unittests (both Expected and AcceptanceTests folder)

Build should be clean with no errors. This might require update of annotations in msrest and msrestazure to solve some of them.
https://github.com/Azure/msrest-for-python/
https://github.com/Azure/msrestazure-for-python/

Swagger comments with \U, \N generate python errors

We have an API that has a description property containing the string:
"Value\Units"

When this description is converted to python, it generates the following string:
"""Value\Units"""

When evaluated, this gives the error:
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 5-6: truncated \UXXXXXXXX escape

I couldn't tell for sure, but I didn't see anything in the OpenAPI definition that precluded the use of \ or \U in a description of an API.
https://swagger.io/specification/
It says:
image

Where common markdown syntax links to here.

Validation code for enum is missing.

It seems when the a property is defined as enum (not x-ms-enum), then no validation code is generated for it. Neither the possible/allowed strings are available as an array or object in the generated code, nor does the validation logic in msrest/serialization.py have something for enum. Is this expected behavior?

Full type hints in models_py3 and aio folder

Update the code generator to have the full annotations on models_py3 and aio folder. Since this is Python 3 files, direct annotations can be used. Some of them are done already.

So roughly this means:

  • Compute the list of necessary typing import (List? Optional? Dict?, Any?) and import just that
  • Annotate correctly the code
  • Be sure that if a class is just needed to type annotation and not direct usage, import it under TYPE_CHECKING global constant

#82 needs to be done before to measure progress and status

Fix Python case

Some Python naming convention are not yet translater correctly to new autorest:
some param names aren't properly converted to python case (they're still capital) (RequiredOptional)

Improve exception generation

Message from @iscai-msft :
error generating HTTP, AzureHead, Paging, AzureSpecials etc (getting schema error when it looks in exception list) -> ask about how the excpetions list has been reformatted

Fix enum comments

For enum service we generated this:

not_applicable = "NotApplicable"  #: User either hasn't started configuring their template 
or they haven't started the configuration process.

instead of

not_applicable = "NotApplicable"  #: User either hasn't started configuring their template or they haven't started the configuration process.

See https://github.com/Azure/azure-sdk-for-python/pull/4366/files

Primitive tests

Support tests about Primitive:

  • BodyBoolean
  • BodyByte
  • BodyDate
  • BodyDateTime
  • BodyDateTimeRfc1123
  • BodyDuration
  • BodyInteger
  • BodyNumber
  • BodyString
  • Header
  • Http
  • Url
  • UrlMultiCollectionFormat

Test infrastructure for Autorest v3

Work on https://github.com/Azure/autorest.python/tree/autorestv3

Short version: port to Python these two calls:

  • gulp regenerate
  • gulp test

Suggestion is to use "invoke": http://www.pyinvoke.org/

gulp regenerate:

Python version in pyinvoke of this gulp script:
https://github.com/Azure/autorest.python/blob/azure-core/.gulp/regeneration.iced

Assuming the testserver has been installed (npm install), this reads the Swaggers and call our current autorest with it to generate in the folders test/vanilla/Expected/AcceptanceTests and test/azure/Expected/AcceptanceTests

gulp test

Python version in pyinvoke of this part script:

autorest.python/.gulp/gulpfile.iced

Lines 23 to 27 in 1d943d2

# Run language-specific tests:
task 'test', "", [], (done) ->
await execute "tox", { cwd: './test/vanilla/' }, defer code, stderr, stdout
await execute "tox", { cwd: './test/azure/' }, defer code, stderr, stdout
done();

IOW, run tox on both vanilla and azure folder

Simplify model generation

The current file structure of generated SDKs creates a "models" folder that contains:

  • One file per paging result
  • One file for all enums
  • One file per model generic Python 2 / 3
  • One file per model specific to Python 3 (which is loaded to replace the previous one if Python 3)

This creates a LOT of files, example with Network:
https://github.com/Azure/azure-sdk-for-python/tree/master/azure-mgmt-network/azure/mgmt/network/v2018_10_01/models

We realized recently that all these tiny files have a massive impacts on performance:

  • It makes the wheel 80% bigger than needed
  • It creates a lot of contention on the I/O (writing small files)
  • Pre-compilation of the package by pip is extremely long
  • It increases the likeliness of long path on Windows issue

Not only performance, but also documentation since it provides two ways to import the same file:

  • import client.models.MyClass
    or
  • import client.models.my_class.MyClass

And actually on Python 3, these are not the same, since the first one includes the automatic choice between Python 3 or Python 2 file (where the second one is Python 2 specific). This creates issues and bugs in the CLI already.

We need to refactor this folder with less files, and fixing the doubling syntax at the same time. This is a breaking changes, and release should upgrade the major version of the resulting package.

I made a prototype of this in Python. This script should be called on generated Autorest folder (loaded in dev mode in a venv) that post-process and patch the files manually. The script is here:
https://gist.github.com/lmazuel/dad0ac92c733b27f6d81ea57a47625ce#file-patch_models_v2-py

The result is:

  • One file for all paging results
  • One file for all enums
  • One file for all models generic Python 2 / 3
  • One file for all models specific to Python 3 (which is loaded to replace the previous one if Python 3)

I released one package already with this patched folder, which can be used as reference:
https://github.com/Azure/azure-sdk-for-python/tree/master/azure-cognitiveservices-vision-contentmoderator/azure/cognitiveservices/vision/contentmoderator/models

CLI already releases using this script to post-process all SDKs (this means CLI doesn't ship the raw SDK but a patched version). We don't have any issues and then assume the prototype works as expected.

The target of this issue is to track backporting the logic of the Python script into Autorest by default.

The tricky part is the inheritance part. Because this new version will declare all classes, but they should be declared in inheritance order, so if B subclass A, A is defined before B in the Python file. The Python script here is doing it using the magic attribute mro, and this logic has to be translated in Autorest core model (which is less intuitive).
Once this part is done, then it's mostly changing the output file of models and is not complicated.

Using not ARM generator, credentials doc says None instead of a link to msrest 

class SpellCheckAPIConfiguration(Configuration):
    """Configuration for SpellCheckAPI
    Note that all parameters used to create this instance are saved as instance
    attributes.
    :param credentials: Subscription credentials which uniquely identify
     client subscription.
    :type credentials: None
    :param str base_url: Service URL
    """

    def __init__(
            self, credentials, base_url=None):

        if credentials is None:
            raise ValueError("Parameter 'credentials' must not be None.")
        if not base_url:
            base_url = 'https://api.cognitive.microsoft.com/bing/v7.0'

Ref: Azure/azure-sdk-for-python#2863

Make --keep-version-file actually interesting

Autorest is supposed to have an option --keep-version-file that act as below:

  • If this is the first generation, just as usual creates a version.py file
  • If a file already exists, keep it as-is and don't touch it

Problem here:

  • I'm not actually sure it works in the first place, it was never tested
  • It exists in the "vanilla" generator, but was never ported to "azure" generator

This issue is about making sure it works as expected, and once it works push it to "azure" as well.

Enum tests

Support tests about:

  • ExtensibleEnums

#161 should be adressed first

Automatic generation of ApiVersion as a query param even when not in Swagger

Not sure if bug or feature request but in my Swagger file I didn't specify any:
"ApiVersionParameter": { "name": "api-version", "in": "query", "required": true, "type": "string", "description": "Client Api Version." }

But in my generated code I still get:
query_parameters['api-version'] = self._serialize.query("self.api_version", self.api_version, 'str')
In every operation, but I don't want it.

I'm using "@microsoft.azure/autorest.python": "^4.0.67", "autorest": "^3.0.5165",

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.