Tag Lookup Help / Feature Request

[still-dreaming-1]

Tag Lookup Help / Feature Request

[masatake]

Do you know "language server" (https://github.com/Microsoft/language-server-protocol) ?
I guess it is what you want.

To realize what you want, we have to think and redesign the interface between editor and ctags.
Currently, tags file is the interface; ctags provides it and an editor reads it. ctags doesn't consider real-time user interaction. I cannot imagine how can I extend ctags to support real-time user interaction. However,
it is obvious to me that, such extended ctags will not be ctags. It will be completely different software.

Some parser of Universal-ctags provides one critical information about context.

int
main(void)
{

}

For the input ctags can generate:

[jet@localhost]~/var/strace% u-ctags --fields=+en -o - /tmp/foo.c 
main	/tmp/foo.c	/^main(void)$/;"	f	line:2	typeref:typename:int	end:5

end: field is newly added in Universal-ctags. With the information, a client tool like editor can know which definition the current line is. As far as I know, there is no editor using this end: information.

[still-dreaming-1]

@masatake That is kind my point though. It seems the very concept of what ctags is, is fundamentally flawed. How is an editor, a tags generator, and a tags file a helpful combination of things? How can a tags file actually be useful to an editor? It is certain not helpful for doing intelligent code definition lookup or intelligent autocompletion.

Language servers are too complex, slow, and buggy. They have to keep running and they save cache information, which is just begging for trouble. All that is fine if it is designed as part of an IDE where all the tools are made by the same team, and they are all designed to work together, but when you try to get a bunch of random tools to work together like this, well I just have never gotten it to work well for something as complex as intelligent code definition lookup or intelligent autocompletion.

Universal ctags is much more simple in that you just run it and wait for the result to be returned/produced after it is finished. It can also generate tags for my code in an instant. Let's say someone wanted to do a code definition lookup. They should be able to tell ctags to generate a code definition lookup tags file. They would pass to ctags the path to the file they are editing, the current contents of the file (which may be different than what is saved, if they haven't saved the latest changes), and a single character representing where the cursor/mouse is at. That character would clue ctags into what they are looking at that they want to find the definition of. Ctags would then use that exact context to generate a very small, one time use file, or potentially just the data itself, that only contains potential candidates for where the thing they are looking up is defined. Ideally it would be so small that it only contains one candidate. That would be the ideal scenario because it means it successfully figured out exactly what they were actually trying to lookup. If it can't narrow it down to a single absolute match, it should attempt to order the results by an estimated likelihood that this is what they are looking for.

Something similar could be done for an autocompletion tags file. Once again, the contextual understanding would need to happen inside of ctags itself, it doesn't make sense for the editor to do that. It can't be that ctags is asked to provide all the methods for a class, it just needs to be asked to provide an autocompletion file, and ctags would be the tool that determines what they are trying to autocomplete. Once again, the user would pass to ctags a path, the current file contents, and the position of the cursor. It would use that information to produce a one time, throw away, autocompletion tags file, or perhaps just the file contents.

The workflow would be different for the editors because they would not be generating a tags file at certain points and then reusing it. They would ask ctags to generate a new one every time they needed to find the definition of something or get autocompletion information.

This would require the logic of ctags to be changed from generating as much information as possible to trying to figure out what is the only information you are looking for based on a context and a goal.

I don't want ctags to become a language server. I don't want it to run in the background and keep cached information. I want it to continue to be an alternative to language servers. I want it to be run to get some results, and then it should stop running until it is ran again to get some different results. It should not reuse old information from the last time it was ran. There are times this will be too slow for certain users or certain code bases, but this design would still help distinguish it from language servers.

[masatake]

An example is needed.

Using the current ^ as the marker that represents the cursor position of an editor.
Think about python as the target language.
Whole source files in a project are already tagged. The results are stored to a tags file.

# funcX is defined in foo
import foo

# funcX is defined in bar, too
import bar

foo.funcX^()

An unintelligent editor may try to search just funcX in the tags file and find two entries: one in foo and the other in bar.

However, your idea is that an ideall tool, let's call it ztags, can do in following steps:

1. The editor passes ztags the positional information of ^ and the name of target source file.
2. ztags makes a query for searching funcX which is defined in a namespace foo.
3. ztags runs the query.
4. ztags returns the result to the editor. The result just for funcX in foo.

Do I understand your intent correctly?
If I'm wrong, please, show me an example with github's markdown notation.

[still-dreaming-1]

That seems exactly right... I guess I'm just hoping the simplicity of doing this with something like ctags would be more reliable than trying to interface with a language server.