Giter Site home page Giter Site logo

aws / codelyzer Goto Github PK

View Code? Open in Web Editor NEW
63.0 17.0 16.0 31.65 MB

Codelyzer is a framework that provides interfaces to build and analyze source code in various languages and generates a platform-independent representation as a universal abstract syntax tree (UAST) model or a JSON file.

License: Apache License 2.0

C# 99.96% F# 0.02% Shell 0.02%
roslyn antlr4 analyzer code source-code

codelyzer's Introduction

Codelyzer

Build Test

Codelyzer is a framework that provides interfaces to build and analyze source code in various languages and generates a platform-independent representation as a universal abstract syntax tree (UAST) model or a JSON file. It offers fine-grained controls to specify the kind of metadata (properties of classes, methods, etc.) to gather and how deep in the hierarchy of the code to search while generating these artifacts. Currently, the framework only supports the C# language.

By generating the output as a JSON file, this framework allows you to develop analysis tools in any language.

Codelyzer - Net

Codelyzer-Net is an analyzer engine for languages based on the Roslyn compiler platform, like C# and VB. The CSharpRoslynProcessor walks an AST to collect metadata of source file components (e.g. solution, projects, namespaces, classes, methods, method invocations, literal expressions, etc). It uses semantic information from a design-time build to collect properties with fully qualified names.

  1. Add the Codelyzer NuGet package source into your Nuget configuration.
  2. Add Codelyzer.Analysis to your project as a Nuget Package.

Getting Started

Follow the example below to see how the library can be integrated into your application for analyzing .NET application.

/* 1. Create logger object */
var loggerFactory = LoggerFactory.Create(builder => 
        builder.SetMinimumLevel(LogLevel.Debug).AddConsole());
var logger = loggerFactory.CreateLogger("Analyzer");
var outputPath = @"/home/users/steve/porting-analysis";

/* 2. Create Configuration settings */
var configuration = new AnalyzerConfiguration(LanguageOptions.CSharp);
configuration.ExportSettings.OutputPath = outputPath;

/* 3. Get Analyzer instance based on language */
var analyzer = CodeAnalyzerFactory.GetAnalyzer(configuration, logger);

/* 4. Analyze the project or solution */
var projectFilePath = @"/home/users/steve/projects/TestProject.csproj";
var analyzerResult = await analyzer.AnalyzeProject(projectFilePath);

Console.WriteLine("The results are exported to file : " + analyzerResult.OutputJsonFilePath);

/* 5. Consume the results as model objects */
var sourcefile = analyzerResult.ProjectResult.SourceFileResults.First();
foreach (var invocation in sourcefile.AllInvocationExpressions())
{
    Console.WriteLine(invocation.MethodName + ":" + invocation.SemanticMethodSignature);
}

var objectCreations = sourcefile.AllObjectCreationExpressions();
var allClasses = sourcefile.AllClasses();
var allMethods = sourcefile.AllMethods();
var allLiterals = sourcefile.AllLiterals();

How to use this code?

  • Clone the Git repository.
  • Load the solution Codelyzer.sln using Visual Studio or Rider.
  • Create a "Run/Debug" Configuration for the "Codelyzer.Analysis" project.
  • Provide command line arguments for a solution and output path, then run the application.

Getting Help

Please use these community resources for getting help. We use the GitHub issues for tracking bugs and feature requests.

Contributing

We welcome community contributions and pull requests. See CONTRIBUTING for information on how to set up a development environment and submit code.

Thank you

License

Libraries in this repository are licensed under the Apache 2.0 License.

See LICENSE and NOTICE for more information.

codelyzer's People

Contributors

aniruddhadave avatar anshuldesai avatar beau-gosse-dev avatar birojnayak avatar cjberg avatar cslong avatar dependabot[bot] avatar dongzw-amz avatar eruanion avatar jiayuw2 avatar jonlouie avatar kanikaul-amazon avatar marknfawaz avatar mingxue008 avatar mrkdeng avatar mrlloyd-amzn avatar neerajhanda avatar nshiramshetti avatar ppittle avatar saikiranakula-amzn avatar shanshanhe avatar xqlin-a avatar xueningl-az avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

codelyzer's Issues

AllInvocationExpressions method returns duplicates of ObjectCreationExpression nodes

Describe the bug

When UstNode.AllInvocationExpressions() is called, any ObjectCreationExpression is returned twice

Expected behavior

Each ObjectCreationExpression should only be returned once.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Analyze a project with at least 1 object expression node
  2. Call AllInvocationExpressions() on a root node
  3. Observe that each object expression node was duplicated

Screenshots

If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information)

  • OS: [e.g. Windows, macOS, Linux, etc]
  • Version/Distro: [e.g. 11, Big Sur, Ubuntu 18.04]

Additional context

Add any other context about the problem here.

Can't get symbol information on certain invocation expressions

Describe the bug

Seems like we can't fetch the symbol information on some of the invocation expression syntax node.

For example
bundles.Add(New ScriptBundle("/bundles/jquery").Include("/Scripts/jquery-{version}.js")) in the VBWebApi Test project.

It might be the case that the inner expressions are causing the lookup in the semantic model to fail.

Expected behavior

Analyzer result should return System.Web.Optimization.BundleCollection.Add(System.Web.Optimization.Bundle), but instead only has the method name Add and Parameters.

Steps to Reproduce

Steps to reproduce the behavior:
Debug using the VBWebApi Test project and break on InvocationExpressionHandler.SetMetaData

Syntax Analysis doesn't run when there are issues with the build environment

There are scenarios where a project fails to build due to environment or configuration issues. In some of these cases, msbuild fails to report back any results preventing a syntax analysis from being generated for the project.

We need to add an option to be able to generate a syntax analysis without depending on the build environment.

exposing metadata settings via CLI

Description

exposing metadata settings via CLI

Solution idea(s)

If some one wants to access Interface declaration, by using CLI he can't get.

add support for semantic method signature

When handling MethodDeclaration nodes, collecting the semantic method signature would provide additional utility, e.g. identify all public method signatures in a project.

Example:

// Method definition
public void Method(StringBuilder sb)
{
}

// Semantic method signature
"public void Method(System.Text.StringBuilder sb)"

Populate 'Parent' property of UstNode objects

Description

  • Parent is a property that exists on the UstNode class but it is unused
  • Populating the Parent property will improve analysis capabilities by preserving the two-way structure of the original syntax tree

Solution idea(s)

A clear and concise description of what you want to happen.

Additional context

Add any other context or screenshots about the feature request here.

Investigate if FullIdentifier values can be retrieved instead of calculated

Description

We use a FullIdentifier for multiple objects (ClassDeclaration, Interface, etc...). This field is the namespace + type + identifier (if any). We need to check if there's a way to get this from Roslyn, instead of us filling ourselves.

Solution idea(s)

Try to get from the roslyn semantic model

Support for reference analysis

Description

Add support for analyzing a solution without building it, using only project references. This will reduce analysis time significantly

Log build errors

When encountering some errors while building, some of the errors are not propagating back to the user. We need a mechanism to read the build log and create a friendly version of these errors

Add project level build support

Description

Add iterator interface for a solution to build project by project

Solution idea(s)

Derive the topo order of project dependencies
Lazy build the projects until it is called by the consumer.

Additional context

Add any other context or screenshots about the feature request here.

Expose function to Read project paths from solution file

Description

We should expose a function to read the project paths from the solution file rather than attempt to discover them through the directory structure of where the solution file is in the codelyzer for all repos to consume.

Solution idea(s)

Exposed function which parses the solution file and returns valid project list.

Additional context

Add any other context or screenshots about the feature request here.

Refactor CSharpRoslynProcessor.cs

There is a lot of duplicated code that can be reused. Here is one example (and there are similar cases for just about every node type):

var identifier = VisitIdentifierName((IdentifierNameSyntax)identifierName);
if (identifier != null)
{
    handler.UstNode.Children.Add(identifier);
}

We can separate these snippets into private methods or an internal helper class using static methods.

Improve unit tests

Need to provide positive and negative unit test cases to detect cases similar to this:
#24 (comment)

These changes will likely result in a shift from our current unit test pattern to one where we implement tests based on file, rather than end-to-end.

Add support for AttributeArguments

Description

Extract information from AttributeArgumentSyntax nodes

Solution idea(s)

A clear and concise description of what you want to happen.

Additional context

Add any other context or screenshots about the feature request here.

Issue with file scoped namespaces in C# 10 and .NET 6

Describe the bug

In C# 10, namespaces can be declared in a new format called file scoped namespaces, which is namespace X.Y.Z; (i.e. with semicolon but no braces). File scoped namespaces (example) are not getting recognized as classes in AllClasses() unless we change the namespace declarations to the old syntax. It would be great to add support for this.

Desktop (please complete the following information)

  • Windows 10 Enterprise
  • Visual Studio 2022

Add support for AssignmentSyntax nodes

Description

  • Currently, Codelyzer can parse operands of an assignment expression but only in isolation
  • Being able to link the left and right operands will give additional analysis capabilities

Solution idea(s)

A clear and concise description of what you want to happen.

Additional context

Add any other context or screenshots about the feature request here.

Hi ๐Ÿ‘‹!

Hey there - maintainer of Buildalyzer here. This project was pointed out by a Buildalyzer user and I thought it was really cool. Glad to see the tool being put to use. I'm also pretty curious about where Codelyzer (cool name ๐Ÿ˜„) is being used (if you can say)?

In any case, don't be a stranger - happy to fix and issues upstream if you find any.

MSBuildDetector is dependent of user's culture

Describe the bug

When MSBuildDetector parse version numbers as decimal numbers (doubles), it doesn't specify the invariant culture. Any user with a culture setting that doesn't use a decimal dot* will result in an error.

*) For instance, Swedish and German cultures use decimal comma (14,1 instead of 14.1).

Expected behavior

MSBuildDetector should be able to parse version numbers independent of a user's current culture setting.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Configure a user culture with a decimal comma, e.g., Set-Culture -CultureInfo 'sv-SE' in PowerShell.
  2. Run unit tests for MSBuildDetector.
  3. Many tests fail.

Screenshots

Failed-tests-2023-04-30

Desktop (please complete the following information)

  • OS: Windows
  • Version/Distro: 10 22H2
  • User culture: Swedish (sv-SE, LCID = 1053)

Additional context

I ran into this problem while trying out Porting Assistant for .NET, that simply crashed and was unusable.

I forked this repo and improved the version parsing. I will create and link a pull request after this issue has been submitted.

IdentifierName bug

Describe the bug

Currently if we use a namespace prefix for any object declaration node, the normally included child node for the identifier name is not present.

Expected behavior

All object declaration nodes should contains an identifier name syntax node as child. (a declaration-node type).

Steps to Reproduce

Use any solution including a prefix namespace in front of your object such as Models.SuperString test = new Models.SuperString(); and examine the codelyzer output not including a declartion-node for the child of this object declaration node.

Screenshots

If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information)

  • OS: [e.g. Windows, macOS, Linux, etc]
  • Version/Distro: [e.g. 11, Big Sur, Ubuntu 18.04]

Additional context

Add any other context about the problem here.

FileNotFoundException observed during LoadMetadataReferences

Describe the bug

LoadMetadataReferences loads project file, extract the file paths referenced in the project file and parse the MetadataReferences from the file paths. We've observed FileNotFoundException being thrown during MetadataReference creation.

Expected behavior

We should first check if the file path referenced in the project file exists, and proceed with MetadataReference creation only when the file exists.

Error logs

"[2021-07-20 02:28:46 ERR] (Porting Assistant IDE Extension) (1.0.2) PortingAssistant.Client.PortingProjectFile.PortingProjectFileHandler: Error while parsing metadata file","System.IO.FileNotFoundException: Could not find a part of the path 'C:\\Users\\<user>\\source\\repos\\<test_project>\\src\\packages\\EntityFramework.6.1.1\\lib\\net45\\EntityFramework.dll'.","File name: 'C:\\Users\\<user>\\source\\repos\\<test_project>\\src\\packages\\EntityFramework.6.1.1\\lib\\net45\\EntityFramework.dll'"," ---> System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\\Users\\<user>\\source\\repos\\<test_project>\\src\\packages\\EntityFramework.6.1.1\\lib\\net45\\EntityFramework.dll'.","   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)","   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)","   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)","   at Roslyn.Utilities.FileUtilities.OpenFileStream(String path)","   at Roslyn.Utilities.FileUtilities.OpenFileStream(String path)","   at Microsoft.CodeAnalysis.MetadataReference.CreateFromFile(String path, MetadataReferenceProperties properties, DocumentationProvider documentation)","   at Codelyzer.Analysis.Build.ProjectBuildHandler.<>c__DisplayClass16_0.<LoadMetadataReferences>b__0(String fileRef)"

Create an analysis without building the project

Description

In some cases, we need to generate an analysis without building the project using just the DLL references. We need to add this feature to codelyzer
The objective is to avoid building the project if we don't have to, which will reduce the time needed for creating an analysis.

Add support for ArgumentList and Argument nodes

We currently ignore arguments of InvocationExpressionSyntax nodes. We should support them since other types of nodes are often defined within arguments, e.g. ObjectCreationExpressionSyntax, LambdaExpressionSyntax.

Bring Codelyzer build results closer to Visual Studio

Description

  • Analyzing this project with Codelyzer results in build errors, but the project builds without errors in Visual Studio and with the MSBuild command.
  • MSBuild commands used:
    • & "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\MSBuild.exe" /p:RestorePackagesConfig=true /restore
    • & "C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Current\Bin\MSBuild.exe" /p:RestorePackagesConfig=true /restore

Solution idea(s)

We need to identify the cause for these build errors and/or the differences between the VS build process and Codelyzer/Buildalyzer's.

Refactor code to make it simpler to access child nodes when analyzing

When running an analysis, we need to simplify the code for accessing child nodes so that a node's children are automatically accessed instead of manually selecting children by adding code to access.

We will still maintain the behavior that any node that isn't handled will not be added to the report.

Generate a graph representation of the code

Description

Create a graph that represents different nodes in the project (Project/class level)

  • Project Level: Creates a graph with nodes that represent Projects and edges representing a dependency on another project
  • Class level: Creates a graph with nodes representing a class/interface and edges representing a relation between two nodes. This relation can be an invocation, declaration, inheritance, member access, etc...

Add a null check to preport syntaxtrees

Describe the bug

When running analysis on a solution with preport references, the syntax tree is sometimes null and causes errors.

Expected behavior

The preport syntax tree shouldn't cause an error when it's null

Steps to Reproduce

Steps to reproduce the behavior:
Run an analysis with a ported project, the issue happens in some instances

Add modifiers to ClassDeclaration nodes

Description

Modifiers are collected for MethodDeclaration nodes. Having them for ClassDeclarations also makes sense and provides additional querying ability.

Solution idea(s)

A clear and concise description of what you want to happen.

Additional context

Add any other context or screenshots about the feature request here.

Add the type of analysis to the result

When semantic analysis fails and a syntactic analysis is performed, the user needs to be notified that this happened so they don't have to parse the error message.

Buildalyzer-Project level build

Description

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Solution idea(s)

A clear and concise description of what you want to happen.

Additional context

Add any other context or screenshots about the feature request here.

Identifiers not getting picked up

Describe the bug

Some identifiers are not getting picked up when they are part of a typed argument list. For example, we are able to analyze IHttpActionResult but not Task

Expected behavior

In both cases above, the identifier should be picked up

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.