Discussion:
Hooking into `make clean` directly?
(too old to reply)
Stephen McDowell
2017-08-30 13:57:49 UTC
Permalink
Hello!

There are no event's for when a `make clean` is fired, are there? I'm
writing an extension that generates a lot of reStructuredText documents
after "builder-inited". I'd like to find a way to enable my users to
simply `make clean` and I do the cleanup. I'm open to advice on any level,
but I stuck with generating the documents automatically for a couple of
reasons

1. Even if I could figure out how to hook directly into sphinx and start
adding some nodes directly (instead of creating files)
a) A large number are made (I'm "recreating" doxygen hierarchies)
b) I'm concerned that (especially for RTD), the AST would no longer fit
in RAM for large projects

2. Generated files allow users to actually see what was made (debug
friendly).

Of course, generating files seems to be a very non-standard thing for an
extension to do ;)

- - - -

Extra info if you're interested:

The setup is basically this: they must write in their index.rst (or
elsewhere) *toctree* directive to incorporate
"contaimentFolder/rootFileName". So I require them to specify these
explicitly to my extensions' arguments in conf.py.

# in conf.py (assuming it is in the extension list)
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst"
}

then say their *index.rst*

.. toctree::
:maxdepth: 2

overview
other_stuff
api/library_root

Since they have to consciously add *api/library_root* in their own docs, I
force them to specify "containmentFolder" and "rootFileName" explicitly (as
well as the title of the document since it's in a toctree).



*Currently*I just emit a warning if they choose "." for the
"containmentFolder". If it is ".", then I prepend the extension name to
every file created to hopefully at least make it clear what was generated.
It's obnoxious though, because now my extension name shows up in their urls.



*Idea*At the very least, maybe I should make the sub-folder a hard
requirement?

Ideally I'd still like to be able to hook into `make clean` directly. If
they remove a class from their API for example, locally their "./api"
folder will start inducing warnings because my extension won't link to it
anymore.

Sorry for the long question, thanks for any insight / suggestions!

-Stephen
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Peter Burdine
2017-08-31 14:50:13 UTC
Permalink
make clean wont' do any of that. That isn't a Sphinx event. That is
handle by the make command and the Makefile script. You can modify the Makefile
script to call other scripts that possibly do what you (eg run a Python
script that exec's conf.py then does the clean up you want).
Post by Stephen McDowell
Hello!
There are no event's for when a `make clean` is fired, are there? I'm
writing an extension that generates a lot of reStructuredText documents
after "builder-inited". I'd like to find a way to enable my users to
simply `make clean` and I do the cleanup. I'm open to advice on any level,
but I stuck with generating the documents automatically for a couple of
reasons
1. Even if I could figure out how to hook directly into sphinx and start
adding some nodes directly (instead of creating files)
a) A large number are made (I'm "recreating" doxygen hierarchies)
b) I'm concerned that (especially for RTD), the AST would no longer
fit in RAM for large projects
2. Generated files allow users to actually see what was made (debug
friendly).
Of course, generating files seems to be a very non-standard thing for an
extension to do ;)
- - - -
The setup is basically this: they must write in their index.rst (or
elsewhere) *toctree* directive to incorporate
"contaimentFolder/rootFileName". So I require them to specify these
explicitly to my extensions' arguments in conf.py.
# in conf.py (assuming it is in the extension list)
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst"
}
then say their *index.rst*
:maxdepth: 2
overview
other_stuff
api/library_root
Since they have to consciously add *api/library_root* in their own docs,
I force them to specify "containmentFolder" and "rootFileName" explicitly
(as well as the title of the document since it's in a toctree).
*Currently*I just emit a warning if they choose "." for the
"containmentFolder". If it is ".", then I prepend the extension name to
every file created to hopefully at least make it clear what was generated.
It's obnoxious though, because now my extension name shows up in their urls.
*Idea*At the very least, maybe I should make the sub-folder a hard
requirement?
Ideally I'd still like to be able to hook into `make clean` directly. If
they remove a class from their API for example, locally their "./api"
folder will start inducing warnings because my extension won't link to it
anymore.
Sorry for the long question, thanks for any insight / suggestions!
-Stephen
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Stephen McDowell
2017-08-31 15:25:36 UTC
Permalink
Yeah, I can include in the docs that they can change their Makefile or something, I was hoping that since newer sphinx do

@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

there was maybe some way to get into `sphinx-build`. Asking people to patch the sphinx-quickstart generated Makefile (given the general complexity) is probably bad advice. Like if they look at it and understand it (because they know `make` syntax and the truly wonderful layout of the generated Makefile), they’ll just patch it themselves ;)

Thanks for your response, I think what I’ll do then is force them to use a sub-directory of app.srcdir and raise an ExtensionError otherwise. This way I can actually just remove that whole tree before regenerating.
make clean wont' do any of that. That isn't a Sphinx event. That is handle by the make command and the Makefile script. You can modify the Makefile script to call other scripts that possibly do what you (eg run a Python script that exec's conf.py then does the clean up you want).
Hello!
There are no event's for when a `make clean` is fired, are there? I'm writing an extension that generates a lot of reStructuredText documents after "builder-inited". I'd like to find a way to enable my users to simply `make clean` and I do the cleanup. I'm open to advice on any level, but I stuck with generating the documents automatically for a couple of reasons
1. Even if I could figure out how to hook directly into sphinx and start adding some nodes directly (instead of creating files)
a) A large number are made (I'm "recreating" doxygen hierarchies)
b) I'm concerned that (especially for RTD), the AST would no longer fit in RAM for large projects
2. Generated files allow users to actually see what was made (debug friendly).
Of course, generating files seems to be a very non-standard thing for an extension to do ;)
- - - -
The setup is basically this: they must write in their index.rst (or elsewhere) toctree directive to incorporate "contaimentFolder/rootFileName". So I require them to specify these explicitly to my extensions' arguments in conf.py.
# in conf.py (assuming it is in the extension list)
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst"
}
then say their index.rst
:maxdepth: 2
overview
other_stuff
api/library_root
Since they have to consciously add api/library_root in their own docs, I force them to specify "containmentFolder" and "rootFileName" explicitly (as well as the title of the document since it's in a toctree).
Currently
I just emit a warning if they choose "." for the "containmentFolder". If it is ".", then I prepend the extension name to every file created to hopefully at least make it clear what was generated. It's obnoxious though, because now my extension name shows up in their urls.
Idea
At the very least, maybe I should make the sub-folder a hard requirement?
Ideally I'd still like to be able to hook into `make clean` directly. If they remove a class from their API for example, locally their "./api" folder will start inducing warnings because my extension won't link to it anymore.
Sorry for the long question, thanks for any insight / suggestions!
-Stephen
--
You received this message because you are subscribed to a topic in the Google Groups "sphinx-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sphinx-dev/y3YnoPJbm6M/unsubscribe <https://groups.google.com/d/topic/sphinx-dev/y3YnoPJbm6M/unsubscribe>.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2017-09-04 16:10:28 UTC
Permalink
Hi,

Unfortunately, there are no events on clean-up process.
Indeed, new Makefile calls sphinx-build on `make clean`. but old
Makefile does not invoke sphinx-build.
So that means Sphinx can't fire on every clean-up.

BTW, `make clean` simply removes build directory. So I think putting
your temporary files on sub directory of builddir. It allows to remove
all files on clean up.

Thanks,
Takeshi KOMIYA
Post by Stephen McDowell
Hello!
There are no event's for when a `make clean` is fired, are there? I'm
writing an extension that generates a lot of reStructuredText documents
after "builder-inited". I'd like to find a way to enable my users to simply
`make clean` and I do the cleanup. I'm open to advice on any level, but I
stuck with generating the documents automatically for a couple of reasons
1. Even if I could figure out how to hook directly into sphinx and start
adding some nodes directly (instead of creating files)
a) A large number are made (I'm "recreating" doxygen hierarchies)
b) I'm concerned that (especially for RTD), the AST would no longer fit
in RAM for large projects
2. Generated files allow users to actually see what was made (debug
friendly).
Of course, generating files seems to be a very non-standard thing for an
extension to do ;)
- - - -
The setup is basically this: they must write in their index.rst (or
elsewhere) toctree directive to incorporate "contaimentFolder/rootFileName".
So I require them to specify these explicitly to my extensions' arguments in
conf.py.
# in conf.py (assuming it is in the extension list)
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst"
}
then say their index.rst
:maxdepth: 2
overview
other_stuff
api/library_root
Since they have to consciously add api/library_root in their own docs, I
force them to specify "containmentFolder" and "rootFileName" explicitly (as
well as the title of the document since it's in a toctree).
Currently
I just emit a warning if they choose "." for the "containmentFolder". If it
is ".", then I prepend the extension name to every file created to hopefully
at least make it clear what was generated. It's obnoxious though, because
now my extension name shows up in their urls.
Idea
At the very least, maybe I should make the sub-folder a hard requirement?
Ideally I'd still like to be able to hook into `make clean` directly. If
they remove a class from their API for example, locally their "./api" folder
will start inducing warnings because my extension won't link to it anymore.
Sorry for the long question, thanks for any insight / suggestions!
-Stephen
--
You received this message because you are subscribed to the Google Groups
"sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Stephen McDowell
2017-09-04 21:50:56 UTC
Permalink
Hi Takeshi,

Yeah I’m definitely not complaining, but I don’t think generating things in the _build directory can be a solution. My understanding is that anything revolving around a toctree (the user must include the “root” document in a toctree somewhere, and I have a large amount of toctree’s document to make sphinx happy / not spit out hundreds of warnings), must be in the source directory.

If toctree’s do not have to be in the source directory, would you be willing to elaborate a little bit? For example, I’m more than willing to implement some (dubious) hacks by changing the AST and/or modifying the list of documents found. I just don’t think that’s a very wise decision in terms of compatibility.

For example, a project with one file that defines one class and one function will have
- 1 file generated for the file
- 1 file generated for the program listing (depending on configs)
- 1 file generated for the directory
- 1 file generated for the class
- 1 file generated for the function

These are all then tied together in a master “library root” document using toctree directives, so that an index is available without dumping the entire output on one page. I was under the impression that the only way to tell Sphinx it needs to process a document outside of the source tree is to use an include directive (e.g. common for people using a shared README.rst for both their repo online and the docs).

If my understanding is correct (include is the only option if I generate everything in _build), as toctree will not allow this, then I think it’s to the birds ;)

Sphinx is probably the most beautiful software I’ve had the privilege of encountering. If I choose to write an extension that is for all intensive purposes ridiculous, that’s on me...

Thanks for all of your responses!

-Stephen
Post by Komiya Takeshi
Hi,
Unfortunately, there are no events on clean-up process.
Indeed, new Makefile calls sphinx-build on `make clean`. but old
Makefile does not invoke sphinx-build.
So that means Sphinx can't fire on every clean-up.
BTW, `make clean` simply removes build directory. So I think putting
your temporary files on sub directory of builddir. It allows to remove
all files on clean up.
Thanks,
Takeshi KOMIYA
Post by Stephen McDowell
Hello!
There are no event's for when a `make clean` is fired, are there? I'm
writing an extension that generates a lot of reStructuredText documents
after "builder-inited". I'd like to find a way to enable my users to simply
`make clean` and I do the cleanup. I'm open to advice on any level, but I
stuck with generating the documents automatically for a couple of reasons
1. Even if I could figure out how to hook directly into sphinx and start
adding some nodes directly (instead of creating files)
a) A large number are made (I'm "recreating" doxygen hierarchies)
b) I'm concerned that (especially for RTD), the AST would no longer fit
in RAM for large projects
2. Generated files allow users to actually see what was made (debug
friendly).
Of course, generating files seems to be a very non-standard thing for an
extension to do ;)
- - - -
The setup is basically this: they must write in their index.rst (or
elsewhere) toctree directive to incorporate "contaimentFolder/rootFileName".
So I require them to specify these explicitly to my extensions' arguments in
conf.py.
# in conf.py (assuming it is in the extension list)
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst"
}
then say their index.rst
:maxdepth: 2
overview
other_stuff
api/library_root
Since they have to consciously add api/library_root in their own docs, I
force them to specify "containmentFolder" and "rootFileName" explicitly (as
well as the title of the document since it's in a toctree).
Currently
I just emit a warning if they choose "." for the "containmentFolder". If it
is ".", then I prepend the extension name to every file created to hopefully
at least make it clear what was generated. It's obnoxious though, because
now my extension name shows up in their urls.
Idea
At the very least, maybe I should make the sub-folder a hard requirement?
Ideally I'd still like to be able to hook into `make clean` directly. If
they remove a class from their API for example, locally their "./api" folder
will start inducing warnings because my extension won't link to it anymore.
Sorry for the long question, thanks for any insight / suggestions!
-Stephen
--
You received this message because you are subscribed to the Google Groups
"sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to a topic in the Google Groups "sphinx-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sphinx-dev/y3YnoPJbm6M/unsubscribe.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...