Completion of error handling

[ohler55]

Completion of error handling

[elfring]

I thank you again that you were willing to adjust this software to some degree (also without direct contribution of financial resources from me so far).
You agreed to reduce return value ignorance in ways that are shown by the published commits.
I imagine that we will reach a software development status where we will stumble on places for which you expressed change reluctance already around specific function calls.

[elfring]

• Agoo is big enough as every useful software. Its source code contains various issues for further development considerations.
  (Our dialogue resulted in some nice adjustments, didn't it?)
• How will our role distribution evolve?
  Do you expect that I (or any other contributor) try to point remaining weaknesses for this issue out so that corresponding improvement possibilities might get more attention (together with traditional programming approaches)?

[ohler55]

• Our dialogs have resulted in nice adjustments.
• How our roles evolve it yet to be seen. I do expect that you and others will continue to point out weaknesses.

[ohler55]

What exactly are you referring to? Can you be more specific in what you are proposing?

[ohler55]

As with any project there will be differences of opinions as well. That should not be taken as an unwillingness to make changes.

[elfring]

I did not understand your proposal.

Can you get into the programming mood to construct object-oriented functionality for C (without a helpful Ruby API)?

Are you trying to say that the software is broken because not all changes proposed are accepted?

Its implementation contains known weaknesses (and open issues).

• How long will it take to fix affected software areas?
• Do you need another reminder for a correction which you applied for an other program already?

[ohler55]

Are you referring to the GNOME Object system? If so then the stated goals of the Object system are not in line with writing Ruby gems nor inline with writing code other people can follow. By the projects own admission the system has "sometimes rather convoluted APIs".

You say the software has know weaknesses but you do not identify what those weaknesses are. If you are referring to the pthread mutex calls then I refute you argument that the software is broken and would say it will not be changed.

[ohler55]

Cool, a code review!

[elfring]

I suggest to avoid ignorance of return values a bit more.
Would you like to detect every error situation as early as possible?

[ohler55]

I believe this has been addressed. Can it be closed?

[elfring]

How will the integration of better source code analysis tools evolve for your software development work flow so that remaining update candidates will be pointed out?

• agoo_log_init
• agoo_malloc
• agoo_queue_multi_init
• graphql_load_file

[ohler55]

I have tried CppCheck with some success but it gives false error warnings enough times that I don't want to or can't really make it an automated part of the build process. It is a useful tool for catching some errors though so I run it sometimes. What tool are you using that detected problems in the functions you mentioned?

As for the functions you pointed out, I made some changes to address what I think you might have been referring to.

agoo_log_init: Added a check for the memory allocation.

agoo_malloc: Added a check for NULL return from malloc. This function is only used in development for checking for memory leaks and writing out of bounds.

agoo_queue_multi_init: Added memory check.

graphql_load_file: added a check for the return value of fseek and ftell. All other errors are being checked and an exception raised on error.

The changes will go into the next release but are currently in the develop branch.

[elfring]

What tool are you using that detected problems in the functions you mentioned?

My own eyes together with GitHub's source code search interface.

As for the functions you pointed out, I made some changes to address what I think you might have been referring to.

Thanks for another software adjustment.
The error detection and corresponding exception handling is still incomplete, isn't it?

[ohler55]

I made some additional changes in debug.c.

[ohler55]

More changes made based on your comments.

[elfring]

I hope that a few of my code review comments will get further constructive responses.

[ohler55]

I believe I have addressed the code review comments with changes in the code.

[elfring]

Partly, yes. - Remaining update candidates will need further software development efforts if you would like to care for these open issues.

[ohler55]

I'm not sure what other update candidates you are referring to.

[elfring]

Function calls for further considerations:

• ALLOC_N
• localtime
• time
• pthread_mutex_lock

[ohler55]

What should be considered on each of those?

ALLOC_N succeeds or raises an exception. Exceptions are currently passed back to the caller.
localtime is supposed to always succeed according to the man pages
time is use once in log.c for a timestamp. There is no other way to get the time so if it fails the timestamp is 1 second before 1970. There is no other way to report the error other than the log itself so any reporting would suffer the same error. Leaving it at one failure that is obvious in the logs is fine.
pthread_mutex_lock we have had this discussion twice now. I've explained my reasons for not adding a check for every lock and unlock. Please accept that I do not intend to change the use of pthread_mutex.

[elfring]

• Where is the identifier “ALLOC_N” defined?
• The three mentioned POSIX functions can generally fail. It will take another while to take this fact better into account also for your software.

[ohler55]

ALLOC_N is a Ruby macro defined in ruby.h.

[ohler55]

Can this be closed?

[elfring]

Not until the discussed exception handling will be complete.

[ohler55]

Anything else?

[elfring]

Obviously, yes.

Further update candidates:

• agoo_pub_write
• response_text

---

Will your software development attention grow for cross-cutting concerns?

[ohler55]

Pushed fixes.

[elfring]

Can you get into the mood to tackle such cross-cutting concerns by any higher level software development means?

[elfring]

I became just curious again if you would like to reconsider previously mentioned information sources once more.
Are you looking for encapsulation of exception handling so that the corresponding software maintenance can become eventually more convenient?

[ohler55]

I have already used CppCheck and run it occasionally. I am not willing to make it part of the CI testing as it gives false positives sometimes. It is useful for periodic checks though. As for Coccinelle, I am comfortable with emacs and using emacs macros to make the kind of changes Coccinelle would help with so I don't see the need to use Coccinelle.

[elfring]

• How do you think about to report unwanted false positives to corresponding software developers?
• I have got doubts about the effectiveness of emacs macros as a generic programming interface for the affected areas.
  Domain-specific languages (like the semantic patch language) can be safer and more convenient for some use cases.
  (SmPL scripts can generate also TODO tags for the Org mode besides direct patches.)

[ohler55]

• I'll consider it but so far have not found the time.
• I don't need a patch language for Agoo. It is not large enough at this point. As for the effectiveness of using emacs, I think that is a personal preference. It has worked well for me for a very long time.

[elfring]

I would appreciate when development tools can help more in reminding for such weaknesses and fixing them.
But I imagine that social factors can hinder progress also in this direction besides general challenges around change willingness and support.

[ohler55]

So can this be closed now?

[elfring]

Not so far because there are still source code places which can look suspicious.

[ohler55]

Why is that suspicious?

Do you intend for this issue to remain open forever? Usually issue are for a specific issue, not a general category like "make it better".

[elfring]

Why is that suspicious?

	for (i = agoo_server.thread_cnt, vp = the_rserver.eval_threads; 0 < i; i--, vp++) {
	    *vp = rb_thread_create(wrap_process_loop, NULL);
	}
	*vp = Qnil;

Can you see a need for another check for the return value from such a function call?

---

Do you intend for this issue to remain open forever?

Not really. - But possible adjustments will depend on the clarification of the running source code analysis and a consensus on review results as usual.

Usually issue are for a specific issue, not a general category like "make it better".

I am unsure on how long it will take to achieve completeness for exception handling here.

[ohler55]

In Ruby the general approach to handling errors is to raise an exception. This is once of those cases. If it fails an exception is raised. There is no need to check at this level. The calling application or the user of the library handles the exception as it sees fit.

Instead of leaving the issue open ended with a vague statement that says handle all exception cases how about identifying the places that you think are problems and when those are fixed the issue is closed. Otherwise with no criteria for completion the issue will have to be rejected as too vague.

[elfring]

If it fails an exception is raised.

• Does this mean that a call of a function from the Ruby C API will not return after an exception was raised (or thrown)?
• Did I overlook any programming interface documentation?

Instead of leaving the issue open ended with a vague statement

Which details do you find too vague here?

that says handle all exception cases

Is this a desirable software development goal?

how about identifying the places that you think are problems

• There are advanced development tools available which can support a corresponding source code search and transformation to some degree.
  (What does hinder you to use them?)

  Remaining examples for update candidates:

  • pthread_mutex_init()
  • pthread_mutex_lock() / pthread_mutex_unlock()

• I suggest to take also additional information sources about program verification better into account.

[ohler55]

• That is correct. A ruby call that raises an exception does not return. Raising an exception does a long jump.
• I don't know what you read about the Ruby C API so I don't know what you missed. It is easy to try though. Write a small app and raise and exception.

• It is a desirable goal to handle all exceptions just like it is desirable to not have any bugs but an issue that says "fix all bugs" isn't very helpful if the bugs are not identified.

• You identified CppCheck as a tool to use and I have used that. It can not be made part of CI as it also gives false positives. I think I have answered this several times now.

• The pthread mutex calls have been been discussed and I have give my reasons for not bothering to check the return codes. The fact that you do not agree with my assessment does not mean the issue should stay open forever. If those are the only identified remaining issue then this can be closed because they have been responded to.

• This comment is a perfect example of a vague issue. Formal verification ranges from minimal using tests on up to a complete mathematical proof the software is correct. Agoo has unit tests and that is the current verification process. A more concise and reasonable issue would be to describe a situation that caused a failure and ask for it to be fixed.

[elfring]

I don't know what you read about the Ruby C API so I don't know what you missed.

Where is the function “rb_thread_create” completely documented?

but an issue that says "fix all bugs" isn't very helpful if the bugs are not identified.

I am proposing the application of a general source code search pattern for a while which can eventually result also in aspect-oriented software development.

I think I have answered this several times now.

Would you like to fix the observation of false positives for discussed analysis tools anyhow?

The pthread mutex calls have been been discussed and I have give my reasons for not bothering to check the return codes.

• I find it strange that you integrated several source code corrections until you would like to block progress in another area of API calls.
• I would interpret such a choice as your programming mistake at the moment.
  🔮 I can only hope that further advices from other contributors might change your mind.

[ohler55]

Sadly, the Ruby C API is not well documented at all or at least I have not found a complete set of API descriptions. The source code is available though.

If you can explain how a "general source code search pattern" would be used and how it would fix some perceived issue then please explain in enough detail so that it can be used.

The false positives for CppCheck are an issue for the CppCheck maintainers but I have not submitted an issue to them yet. Not having submitted an issue is not an Agoo issue though.

I accepted your suggestions on issues where I agreed with you. That does not mean I will agree with you on all things you think are an issue. You can interpret that disagreement as a mistake as you wish but it would be nice if you would respect my opinion as well when we disagree.

[ohler55]

So here is the deal. Provide a clear well defined issue that can be addressed or this issue will be closed. There are some helpful guidelines on issue submission that describe identify some good practices. Please take a look at https://github.com/magento/magento2/wiki/Issue-reporting-guidelines and/or https://github.com/necolas/issue-guidelines. Issues submissions that follow those guidelines I can address either by fixing or rejecting as not relevant.

[elfring]

Sadly, the Ruby C API is not well documented at all

Would we like to influence this software situation anyhow?

If you can explain how a "general source code search pattern" would be used

I am used to specific software weakness searches for years as you can determine such experiences from my public bug reports.
Do you need any additional training in the mentioned directions?

and how it would fix some perceived issue then please explain in enough detail so that it can be used.

Other information sources explained this already in sufficient ways.
It seems that you stumble on general challenges to integrate known advices from there.

… but I have not submitted an issue to them yet.

Would you dare to point further development considerations out to the maintainers of affected analysis and code transformation tools?

That does not mean I will agree with you on all things you think are an issue.

This view is fine in principle.

… but it would be nice if you would respect my opinion

• I can respect some of your opinions.
• Specific disagreements are influenced by various factors.
• I dared to remind you about questionable views (while they can be connected with reasonable development concerns) because they reduce the desired software correctness.
• Further software evolution can eventually happen in discussed areas together with additional incentives.

Provide a clear well defined issue that can be addressed

• Who would like to improve the situation around calls from the POSIX (and Ruby) thread programming interface?
• Would you like to continue the clarification for each API function call on a separate issue?

[ohler55]

Ruby is open source so anyone can contribute to it. If by influence you mean I should document the APIs then I am not ready to spend my time on that effort. If you mean thank someone else for doing it then that I can do.

You have pointed out a number of places where the code can be improved. I thank you for that. As for training in finding areas that can be improved I am always interested in improving my development skills but I am not interested in searching through other peoples code for weaknesses. I prefer to develop code instead.

I would certainly prefer to have each issue clearly defined. For example, if you were to create an issue indicating return values of pthread_mutex functions are not checked and could lead to concurrency issues that would be a reasonable issue. I would reject it with a similar description that I have given before but that is the preferred granularity and clarity for an issue. Give me something I can work with.

Another example might be around malloc use. I have already gone over the code and think all the return values from malloc are checked but I may have missed one somewhere. You could say, the malloc return value in file xxx.c line 123 is not checked or list the files and lines if there are multiple occurrences. You created a separate issue for the signal handler. That was perfect. A clear problem and discernible goal and success criteria.

[elfring]

If by influence you mean I should document the APIs …

I am just curious who might become motivated enough to adjust the source code of the documentation another bit for the involved software.

… skills but I am not interested in searching through other peoples code for weaknesses.

This information seems to indicate another significant difference between us.

I prefer to develop code instead.

We tend to distribute our software development capacity also in different ways.

I would certainly prefer to have each issue clearly defined.

It seems that I need to interpret such a wording as another indication from your personal variation of change resistance (which might have grown for your software components the more I pointed open issues out).

Give me something I can work with.

Repetition: pthread_mutex_init ⇒ agoo_server_setup

[ohler55]

I am happy to make changes to fix issues. I think I have demonstrated that as I made a lot of changes related to this issue. You will note that each set of changes was prompted by a clearer description of that part of this issue. I think you will find most projects expect a description of what should be fixed that does not change as and expand as each change is made. Anyway lets close this and you can open up more explicit issues if you find something that you think should be addressed.

[elfring]

I am happy to make changes to fix issues.

Can such a view trigger activities for inspiration and help with additional evolution in related areas?

You will note that each set of changes was prompted by a clearer description of that part of this issue.

We came along different expectations for clearness.

Anyway lets close this

You should know also from our development discussion where this software implementation is still unfinished (and therefore improvable) so far, shouldn't you?

I guess that you are used to some tools for working with (regular) search patterns.
When will similar search patterns be specified as pointcuts and corresponding advice (according to aspect-oriented software development)?

and you can open up more explicit issues if you find something that you think should be addressed.

Under which circumstances would you consider the application of scoped locks here?

[ohler55]

Software is never finished. There is always something that can be done to improve or add new features.

I am familiar with some of the search too such as grep and have written my own helper utilities as well.

I have not seen the need for scoped locks. They are a relatively new feature of C++ so not really something I plan on using in a C only application even if there was a need for them.

[elfring]

They are a relatively new feature of C++ so not really something I plan on using in a C only application even if there was a need for them.

• Are scoped locks an useful programming tool for fixing a few more places in your software?
• Would you like to stick to std::lock_guard objects for a while?
• Do you prefer Ruby programming interfaces for the same functionality?

[ohler55]

They are not needed. There is no fix needed that I am aware of.

No, I do not want to use any C++ lock features.

There is no choice in Ruby. The GVL must be used for the Ruby portion.

[elfring]

There is no fix needed that I am aware of.

I pointed out that there are return values still inappropriately ignored from calls of at least three functions from the POSIX API.
I am curious about the circumstances for possible fixes there.

[ohler55]

If you are referring to the pthread mutex calls again the forget it. I am not willing to convert to C++ to use a scoped guard when what is currently in place works fine and works on compilers older than the most recent C++ compiler.

[elfring]

Would you like to emulate the combination of constructors and destructors for the C programming language by using perilogues?

[ohler55]

I'll look into it.

[elfring]

How do you think about the possibility of simulating failures for POSIX API function calls in your test system?

[ohler55]

I fail to see how perilogue has anything to do with the C code in this project.

[ohler55]

Simulating failures would be interesting. I'm not sure it would help with the majority of issues that arise in reality though. Running out of memory is not a common failure scenario in my experience.

[elfring]

How do you think about to call functions over pointer combinations from specific data structures so that the actions “construction”, “main task execution” and “destruction” will be safely performed?

---

• Will any web server users become interested in the error behaviour of your software if the desired mutual exclusion will “accidentally” fail anyhow?
• Would you eventually like to care more for data processing invariants?

[ohler55]

I'm not sure what you mean by the constructor and destructor comment. I already us an approach for object base functions. Is that what you mean?

Of course I would like to handle errors well. Failures are unlikely to be from out of memory though. If there was a tool to change return values that might be useful.

I was not able to understand your question about invariants.

[elfring]

I'm not sure what you mean by the constructor and destructor comment.

This functionality to which we are used in object-oriented programming languages (like C++ and Ruby) can be provided also by C code.
Would you like to fiddle with any function pointer lists?

Of course I would like to handle errors well.

How can your remaining change resistance be adjusted für this purpose?

If there was a tool to change return values that might be useful.

How do you think about to use programs together with the software “libfiu”?

I was not able to understand your question about invariants.

Please read the corresponding literature.

[ohler55]

I am happy with my current approach that used function names than end in _create or _destroy. I see no need to change the names to be constructor and destructor is that is what you are referring to.

You assume I am resistant to change. I think I have demonstrated I am not so your assumption is wrong.

As for libfiu, I am not ready to add the complexity of using that library nor willing to force Ruby users to have to install the fiu library when using the agoo gem.

[elfring]

I see no need to change the names to be constructor and destructor is that is what you are referring to.

I suggest to consider additional software design options (even if you prefer to restrict the solution space considerably).

I think I have demonstrated I am not

You published some useful software updates.

so your assumption is wrong.

I am trying for a while to adjust general communication difficulties.

You started to block specific change possibilities according to the view that I would eventually not respect different software development opinions.

As for libfiu, …

How do you think about to use programs like “fiu-run” and “fiu-ctrl” for your own test approaches (when you find other code analysis tools not helpful enough so far)?

[ohler55]

Instead of say some like "consider additional software design options" how about being more explicit about the design options you are referring to.

Willing to consider changes does not mean all proposed changes will be accepted.

flu-run and fiu-ctrl require that that the application use libfiu. As I explained, that is no something I am willing to force on users of agoo.

[elfring]

how about being more explicit about the design options you are referring to.

Do you understand my proposal to call function pointer combinations so that object-oriented functionality will be achieved for C code because you would like to stick to this programming language (besides a Ruby API)?

Willing to consider changes does not mean all proposed changes will be accepted.

• Such a view is generally fine.
• Can it affect progress for fixing correctness also for your software components?

As I explained, that is no something I am willing to force on users of agoo.

It seems that you interpreted my program suggestion in an unwanted direction.
Additional tools can show helpful software development opportunities for interested parties, can't they?

[ohler55]

I did not understand your proposal. If you look at the code you will see there are some places where I do use function pointers as suggested. It is a technique I have used when it seemed appropriate.

• I can't make sense of this sentence. Are you trying to say that the software is broken because not all changes proposed are accepted?

Additional tool can be useful. Be explicit about the tool and it will be considered. librfiu was considered and rejected. CppCheck was considered and used.

[elfring]

You say the software has know weaknesses but you do not identify what those weaknesses are.

Please check the available facts again.

If you are referring to the pthread mutex calls then I refute you argument that the software is broken and would say it will not be changed.

Does this feedback fit to my analysis and interpretation of a questionable software situation because of remaining change resistance?

[ohler55]

Obviously this is not going anywhere. You are stuck on the pthread mutex calls and claim that if they are not changed the way you want the software is broken. I disagree. Since we can not agree this supposed issue is dead. It is being closed. If you wish to open a new issue for a clearly define problem feel free to do so.