Discussion:
Problem translating non-RST source file
(too old to reply)
Matthias Geier
2018-01-03 15:14:48 UTC
Permalink
Dear Sphinx Hackers.

I have no clue about translating Sphinx documents, but I've gotten a
related bug report for my Sphinx source parser extension "nbsphinx":

https://github.com/spatialaudio/nbsphinx/issues/154

I don't know how the Sphinx translation machinery works, is it
supposed to work on non-RST source files?

Is it working at the level of reST source text or is is working on the
docutils representation?

What can I do with my extension to make translations work?

Or is translation only working for RST files?

cheers,
Matthias
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
gilberto dos santos alves
2018-01-04 01:43:14 UTC
Permalink
i will make tests with pt_br language. proif of concept for understand if
issue is related with encoded utf8 files , gettext or sphinx or whatelse we
discover!
make sure that all your files have directive python for utf-8 encode and
sphinx config too.
Post by Matthias Geier
Dear Sphinx Hackers.
I have no clue about translating Sphinx documents, but I've gotten a
https://github.com/spatialaudio/nbsphinx/issues/154
I don't know how the Sphinx translation machinery works, is it
supposed to work on non-RST source files?
Is it working at the level of reST source text or is is working on the
docutils representation?
What can I do with my extension to make translations work?
Or is translation only working for RST files?
cheers,
Matthias
--
You received this message because you are subscribed to the Google Groups
"sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-04 12:10:52 UTC
Permalink
Hi,
Post by Matthias Geier
I don't know how the Sphinx translation machinery works, is it
supposed to work on non-RST source files?
The answer is yes and no. The parser should be able to parse a part of
document (a.k.a paragraph).
In this case, nbsphinx does not support it.

AFAIK, there are no docs for responsibility of source parsers. So this
is implicit requirements.
I guess nobody had not be able to suppose it on enhancing source parsers.
Post by Matthias Geier
Is it working at the level of reST source text or is is working on the
docutils representation?
Sphinx i18n feature uses latter way.

1. i18n processor (sphinx.tranforms.i18n.Locale) extracts raw source
text from translatable nodes.
2. It translates them using message catalog (.mo files) one by one
3. It invokes the parser and obtains parsed doctree from translated
message (also one by one)
4. It substitutes the original message node with translated one.


Thanks,
Takeshi KOMIYA
Post by Matthias Geier
Dear Sphinx Hackers.
I have no clue about translating Sphinx documents, but I've gotten a
https://github.com/spatialaudio/nbsphinx/issues/154
I don't know how the Sphinx translation machinery works, is it
supposed to work on non-RST source files?
Is it working at the level of reST source text or is is working on the
docutils representation?
What can I do with my extension to make translations work?
Or is translation only working for RST files?
cheers,
Matthias
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Matthias Geier
2018-01-05 11:48:33 UTC
Permalink
Hi Komiya-san.
Post by Komiya Takeshi
Hi,
Post by Matthias Geier
I don't know how the Sphinx translation machinery works, is it
supposed to work on non-RST source files?
The answer is yes and no. The parser should be able to parse a part of
document (a.k.a paragraph).
In this case, nbsphinx does not support it.
OK, I see.
nbsphinx does indeed not support this, but I'm wondering if I can make
this work.

I guess the relevant function is

SomeParser.parse(self, inputstring, document) -> None

Currently my parser expects the "inputstring" to be a JSON string
holding the Jupyter notebook contents.

Is there a way that I can know from within the parse() function
whether I'm called with the string of a full source file or with just
a snippet?

If not, I can probably just add a heuristic to check if I'm dealing
with a JSON string or not (e.g. "does it start with an opening
brace?").
Or is this a bad idea?
Post by Komiya Takeshi
AFAIK, there are no docs for responsibility of source parsers. So this
is implicit requirements.
I guess nobody had not be able to suppose it on enhancing source parsers.
I guess this makes sense and for many parser it doesn't make a
difference if they get a full document or just a paragraph.

In my case it does make a difference, though.
Post by Komiya Takeshi
Post by Matthias Geier
Is it working at the level of reST source text or is is working on the
docutils representation?
Sphinx i18n feature uses latter way.
1. i18n processor (sphinx.tranforms.i18n.Locale) extracts raw source
text from translatable nodes.
2. It translates them using message catalog (.mo files) one by one
3. It invokes the parser and obtains parsed doctree from translated
message (also one by one)
4. It substitutes the original message node with translated one.
OK, thanks for the explanation, that's really good to know!

cheers,
Matthias
Post by Komiya Takeshi
Thanks,
Takeshi KOMIYA
Post by Matthias Geier
Dear Sphinx Hackers.
I have no clue about translating Sphinx documents, but I've gotten a
https://github.com/spatialaudio/nbsphinx/issues/154
I don't know how the Sphinx translation machinery works, is it
supposed to work on non-RST source files?
Is it working at the level of reST source text or is is working on the
docutils representation?
What can I do with my extension to make translations work?
Or is translation only working for RST files?
cheers,
Matthias
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-05 13:37:04 UTC
Permalink
Hi,
Post by Matthias Geier
Is there a way that I can know from within the parse() function
whether I'm called with the string of a full source file or with just
a snippet?
Unfortunately, there are no way. In docutils, parser class can't know
where is called from.
Post by Matthias Geier
If not, I can probably just add a heuristic to check if I'm dealing
with a JSON string or not (e.g. "does it start with an opening
brace?").
Or is this a bad idea?
Surely it is not good way, but it is only way to handle translation as
far as I know.

Thanks,
Takeshi KOMIYA
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-06 04:14:39 UTC
Permalink
Hi Gunter,
As Sphinx uses Docutils for rST processing, the documentation in
http://docutils.sourceforge.net/docs/ may help.
Indeed Sphinx uses docutils' parser for its source parsers. So its
document is useful for developers.
But its responsibility is not same between docutils and Sphinx.
I meant no document describes it.

In Sphinx, source parsers should parse both whole of document and a
fragment of it to realize translation.
Usually they are same process for almost of source parsers.
But, in this case, nbsphinx should parse Jupyter notebook (JSON doc)
and a paragraph written in reST or markdown.
Because Jupyter notebook is a kind of container.

Takeshi KOMIYA
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Matthias Geier
2018-01-08 14:02:16 UTC
Permalink
Thanks Günter and Takeshi for your help!

I've created a PR implementing the check for a valid JSON description
of a Jupyter notebook:
https://github.com/spatialaudio/nbsphinx/pull/156

I think this should work now, but it is going to be a temporary
solution. For now I'm using an intermediate reST representation,
therefore the translation strings also use reST markup.
Some time in the future (as soon as an extensible CommonMark library
for Python is available), I'll try to get rid of this intermediate
reST stage and directly parse into the docutils representation.
At this time, I guess the translation strings will change from reST to
CommonMark.
Post by Komiya Takeshi
Hi Gunter,
As Sphinx uses Docutils for rST processing, the documentation in
http://docutils.sourceforge.net/docs/ may help.
Indeed Sphinx uses docutils' parser for its source parsers. So its
document is useful for developers.
But its responsibility is not same between docutils and Sphinx.
I meant no document describes it.
Yes, currently developers need to read both, Sphinx and Docutils
documentation and combine them to get the full picture.
Post by Komiya Takeshi
In Sphinx, source parsers should parse both whole of document and a
fragment of it to realize translation.
Usually they are same process for almost of source parsers.
The division of duties in Docutils means that the `parser` component is
ignorant about the source of the string it parses.
It is the duty of the `reader` component to extract the to-be-translated
rST source out of the context (a complete rST document, an rST document
fraction, doctrings of a Python file, multiple documents, ...) and hand
it as a string to the `parser`.
Post by Komiya Takeshi
But, in this case, nbsphinx should parse Jupyter notebook (JSON doc)
and a paragraph written in reST or markdown.
Because Jupyter notebook is a kind of container.
Yes, and therefore the nbspinx `reader` component must care for handling
the various cases (complete Jupyter notebook vs. source text without
wrapper) accordingly.
Are you suggesting that I should implement such a custom "reader" component?
How would this work together with Sphinx?
Do you happen to know another third-party library that did a similar thing?
Depending on the source format, either the `rST parser` or the `markdown
parser` should be called to parse the source string.
The translation then works on the "document tree" returned by the parser.
OK, that makes sense.

cheers,
Matthias
Günter
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-08 14:51:43 UTC
Permalink
Post by Matthias Geier
(as soon as an extensible CommonMark library
for Python is available)
Personally, I'm developing CommonMark parser for docutils; pycmark.
It has module structure. So you will add or remove notation with API.
The goal of pycmark is providing good GFM parser for docutils and Sphinx.
https://github.com/tk0miya/pycmark

It is not released first alpha version yet. But I suggest it for you
if completed :-)
Post by Matthias Geier
Are you suggesting that I should implement such a custom "reader" component?
How would this work together with Sphinx?
I don't have idea for this problem yet, but I think we need to enhance
Sphinx framework to translate this kind of document.
I think Jupyter notebook is a container format which contains other
document formats (like reST, markdown and so on).
Hence we need to switch parsers for whole of document and each fragments.

For example, nbsphinx give a hint for Sphinx like following:

<document>
<paragraph format="reST">
blah blah blah
<paragraph format="markdown">
blah blah blah
<figure>
<caption format="reST">
blah blah blah
<image>

I don't know this is good for our case or not. But it can help
sphinx-core to switch parsers.

Thanks,
Takeshi KOMIYA
Post by Matthias Geier
Thanks Günter and Takeshi for your help!
I've created a PR implementing the check for a valid JSON description
https://github.com/spatialaudio/nbsphinx/pull/156
I think this should work now, but it is going to be a temporary
solution. For now I'm using an intermediate reST representation,
therefore the translation strings also use reST markup.
Some time in the future (as soon as an extensible CommonMark library
for Python is available), I'll try to get rid of this intermediate
reST stage and directly parse into the docutils representation.
At this time, I guess the translation strings will change from reST to
CommonMark.
Post by Komiya Takeshi
Hi Gunter,
As Sphinx uses Docutils for rST processing, the documentation in
http://docutils.sourceforge.net/docs/ may help.
Indeed Sphinx uses docutils' parser for its source parsers. So its
document is useful for developers.
But its responsibility is not same between docutils and Sphinx.
I meant no document describes it.
Yes, currently developers need to read both, Sphinx and Docutils
documentation and combine them to get the full picture.
Post by Komiya Takeshi
In Sphinx, source parsers should parse both whole of document and a
fragment of it to realize translation.
Usually they are same process for almost of source parsers.
The division of duties in Docutils means that the `parser` component is
ignorant about the source of the string it parses.
It is the duty of the `reader` component to extract the to-be-translated
rST source out of the context (a complete rST document, an rST document
fraction, doctrings of a Python file, multiple documents, ...) and hand
it as a string to the `parser`.
Post by Komiya Takeshi
But, in this case, nbsphinx should parse Jupyter notebook (JSON doc)
and a paragraph written in reST or markdown.
Because Jupyter notebook is a kind of container.
Yes, and therefore the nbspinx `reader` component must care for handling
the various cases (complete Jupyter notebook vs. source text without
wrapper) accordingly.
Are you suggesting that I should implement such a custom "reader" component?
How would this work together with Sphinx?
Do you happen to know another third-party library that did a similar thing?
Depending on the source format, either the `rST parser` or the `markdown
parser` should be called to parse the source string.
The translation then works on the "document tree" returned by the parser.
OK, that makes sense.
cheers,
Matthias
Günter
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Matthias Geier
2018-01-09 17:24:37 UTC
Permalink
Post by Komiya Takeshi
Post by Matthias Geier
(as soon as an extensible CommonMark library
for Python is available)
Personally, I'm developing CommonMark parser for docutils; pycmark.
It has module structure. So you will add or remove notation with API.
The goal of pycmark is providing good GFM parser for docutils and Sphinx.
https://github.com/tk0miya/pycmark
It is not released first alpha version yet. But I suggest it for you
if completed :-)
That looks great!
Based on a quick look it seems to have the kind of extensibility I'm
looking for.

Did you already do some speed comparisons with other CommonMark implementations?

Are you able to create simple HTML output compatible to the spec
(https://github.com/commonmark/CommonMark/blob/master/spec.txt)?
I've seen that you've manually created many test cases. Those would
become obsolete if you could simply read spec.txt for your tests.

This would also simplify testing extensions, not only for core CommonMark.
Post by Komiya Takeshi
Post by Matthias Geier
Are you suggesting that I should implement such a custom "reader" component?
How would this work together with Sphinx?
I don't have idea for this problem yet, but I think we need to enhance
Sphinx framework to translate this kind of document.
I think Jupyter notebook is a container format which contains other
document formats (like reST, markdown and so on).
Hence we need to switch parsers for whole of document and each fragments.
<document>
<paragraph format="reST">
blah blah blah
<paragraph format="markdown">
blah blah blah
<figure>
<caption format="reST">
blah blah blah
<image>
I don't know this is good for our case or not. But it can help
sphinx-core to switch parsers.
Yes, something like this would be the cleanest way to do it for
Jupyter notebooks.

However, I'm not sure how important this feature is compared to the
complexity it would add to Sphinx.

I assume the huge majority of cases in Jupyter notebooks involves
Markdown cells.
Only a very small part will use raw reST cells or reST output cells.
And then it would still only be relevant if those are meant to be
translated to another language.
Basically all non-typewriter text visible in the Jupyter Notebook is Markdown.
The only exception I can think of would be in the *output* of code
cells, but I think this isn't supposed to be translated anyway, right?
Same as one wouldn't want to translate the input source code of code
cells, I suppose.

BTW, are source code comments supposed to be translatable?

I guess we could live with assuming Markdown snippets for the
translation strings, but if you think a per-snippet parser selection
would be a meaningful addition to Sphinx core, I would of course like
to use it in nbsphinx.

Would it make sense in any other scenario?

For example, I could imagine a custom "markdown" directive as a reST
extension, something like this::

Some `reST <http://example.com/rst/>`_ text.

.. markdown::

Some [Markdown](http://example.com/md/) text.

And more :code:`reST` text.

I don't know if anyone would need such a thing, though.

BTW, the code in the "Locale" class in sphinx/transforms/i18n.py seems
to be pretty source-parser-agnostic except some parts mentioning "::"
which is reST-specific if I'm not mistaken. But I guess it doesn't
really hurt on other source parsers.

cheers,
Matthias
Post by Komiya Takeshi
Thanks,
Takeshi KOMIYA
Post by Matthias Geier
Thanks Günter and Takeshi for your help!
I've created a PR implementing the check for a valid JSON description
https://github.com/spatialaudio/nbsphinx/pull/156
I think this should work now, but it is going to be a temporary
solution. For now I'm using an intermediate reST representation,
therefore the translation strings also use reST markup.
Some time in the future (as soon as an extensible CommonMark library
for Python is available), I'll try to get rid of this intermediate
reST stage and directly parse into the docutils representation.
At this time, I guess the translation strings will change from reST to
CommonMark.
Post by Komiya Takeshi
Hi Gunter,
As Sphinx uses Docutils for rST processing, the documentation in
http://docutils.sourceforge.net/docs/ may help.
Indeed Sphinx uses docutils' parser for its source parsers. So its
document is useful for developers.
But its responsibility is not same between docutils and Sphinx.
I meant no document describes it.
Yes, currently developers need to read both, Sphinx and Docutils
documentation and combine them to get the full picture.
Post by Komiya Takeshi
In Sphinx, source parsers should parse both whole of document and a
fragment of it to realize translation.
Usually they are same process for almost of source parsers.
The division of duties in Docutils means that the `parser` component is
ignorant about the source of the string it parses.
It is the duty of the `reader` component to extract the to-be-translated
rST source out of the context (a complete rST document, an rST document
fraction, doctrings of a Python file, multiple documents, ...) and hand
it as a string to the `parser`.
Post by Komiya Takeshi
But, in this case, nbsphinx should parse Jupyter notebook (JSON doc)
and a paragraph written in reST or markdown.
Because Jupyter notebook is a kind of container.
Yes, and therefore the nbspinx `reader` component must care for handling
the various cases (complete Jupyter notebook vs. source text without
wrapper) accordingly.
Are you suggesting that I should implement such a custom "reader" component?
How would this work together with Sphinx?
Do you happen to know another third-party library that did a similar thing?
Depending on the source format, either the `rST parser` or the `markdown
parser` should be called to parse the source string.
The translation then works on the "document tree" returned by the parser.
OK, that makes sense.
cheers,
Matthias
Günter
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-17 16:29:40 UTC
Permalink
Post by Matthias Geier
That looks great!
Based on a quick look it seems to have the kind of extensibility I'm
looking for.
Thanks :-)
I hope it will help you.
Post by Matthias Geier
Did you already do some speed comparisons with other CommonMark implementations?
Not yet. I just finished to parse almost of specs. But there are some
remaining works (ex. TABs, unicode spacings and so on).
In addition, I have to make transforms to adjust doctree for docutils writers.

Now I stopped the work because I'm in busy for Sphinx.
Post by Matthias Geier
Are you able to create simple HTML output compatible to the spec
(https://github.com/commonmark/CommonMark/blob/master/spec.txt)?
I've seen that you've manually created many test cases. Those would
become obsolete if you could simply read spec.txt for your tests.
Yes, my test cases are used for development. I have to use
spec_tests.py of commonmark to confirm it works well.
It is one of my remaining work.
Post by Matthias Geier
Yes, something like this would be the cleanest way to do it for
Jupyter notebooks.
However, I'm not sure how important this feature is compared to the
complexity it would add to Sphinx.
I guess some extensions expects the feature.
For example, autodoc for other languages might require to parse markdown texts.

I think there are some kind of text container formats.
To support them simply, it would be nice if we can switch parsers easily.

I know this is tough work. So I can't promise when it is introduced.
Post by Matthias Geier
I assume the huge majority of cases in Jupyter notebooks involves
Markdown cells.
Only a very small part will use raw reST cells or reST output cells.
And then it would still only be relevant if those are meant to be
translated to another language.
Basically all non-typewriter text visible in the Jupyter Notebook is Markdown.
That's right. Markdown (especially GFM) is major and important mark up format.
So to support it is also very important for Sphinx.
Post by Matthias Geier
The only exception I can think of would be in the *output* of code
cells, but I think this isn't supposed to be translated anyway, right?
Yes, literal_nodes are skipped.
Post by Matthias Geier
BTW, are source code comments supposed to be translatable?
Also yes.
Post by Matthias Geier
BTW, the code in the "Locale" class in sphinx/transforms/i18n.py seems
to be pretty source-parser-agnostic except some parts mentioning "::"
which is reST-specific if I'm not mistaken. But I guess it doesn't
really hurt on other source parsers.
I've never read i18n.py, so I can't say anything. But it has been used
for a long time.
So it might not expect other source parsers.

Thanks,
Takeshi KOMIYA
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Matthias Geier
2018-01-22 13:01:13 UTC
Permalink
Post by Komiya Takeshi
Post by Matthias Geier
That looks great!
Based on a quick look it seems to have the kind of extensibility I'm
looking for.
Thanks :-)
I hope it will help you.
Post by Matthias Geier
Did you already do some speed comparisons with other CommonMark implementations?
Not yet. I just finished to parse almost of specs. But there are some
remaining works (ex. TABs, unicode spacings and so on).
In addition, I have to make transforms to adjust doctree for docutils writers.
Now I stopped the work because I'm in busy for Sphinx.
OK, that's unfortunate, but I also really appreciate your work on Sphinx!
Post by Komiya Takeshi
Post by Matthias Geier
Are you able to create simple HTML output compatible to the spec
(https://github.com/commonmark/CommonMark/blob/master/spec.txt)?
I've seen that you've manually created many test cases. Those would
become obsolete if you could simply read spec.txt for your tests.
Yes, my test cases are used for development. I have to use
spec_tests.py of commonmark to confirm it works well.
It is one of my remaining work.
Post by Matthias Geier
Yes, something like this would be the cleanest way to do it for
Jupyter notebooks.
However, I'm not sure how important this feature is compared to the
complexity it would add to Sphinx.
I guess some extensions expects the feature.
For example, autodoc for other languages might require to parse markdown texts.
Oh yes, autodoc with Markdown is a good example!
Post by Komiya Takeshi
I think there are some kind of text container formats.
To support them simply, it would be nice if we can switch parsers easily.
Yes that sounds reasonable.
Post by Komiya Takeshi
I know this is tough work. So I can't promise when it is introduced.
I'm looking forward to it, but I understand that it may take a while ...
Post by Komiya Takeshi
Post by Matthias Geier
I assume the huge majority of cases in Jupyter notebooks involves
Markdown cells.
Only a very small part will use raw reST cells or reST output cells.
And then it would still only be relevant if those are meant to be
translated to another language.
Basically all non-typewriter text visible in the Jupyter Notebook is Markdown.
That's right. Markdown (especially GFM) is major and important mark up format.
So to support it is also very important for Sphinx.
Post by Matthias Geier
The only exception I can think of would be in the *output* of code
cells, but I think this isn't supposed to be translated anyway, right?
Yes, literal_nodes are skipped.
OK, I thought so.
Post by Komiya Takeshi
Post by Matthias Geier
BTW, are source code comments supposed to be translatable?
Also yes.
Yes?
Don't you mean "also no"?
Post by Komiya Takeshi
Post by Matthias Geier
BTW, the code in the "Locale" class in sphinx/transforms/i18n.py seems
to be pretty source-parser-agnostic except some parts mentioning "::"
which is reST-specific if I'm not mistaken. But I guess it doesn't
really hurt on other source parsers.
I've never read i18n.py, so I can't say anything. But it has been used
for a long time.
So it might not expect other source parsers.
OK, I see.

Keep up the good work!

cheers,
Matthias
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-23 06:49:21 UTC
Permalink
Post by Matthias Geier
Post by Komiya Takeshi
Post by Matthias Geier
BTW, are source code comments supposed to be translatable?
Also yes.
Yes?
Don't you mean "also no"?
Oh, sorry. It was my wrong. comments are not translated.
I'd made a mistake on reading source code.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Komiya Takeshi
2018-01-08 14:38:59 UTC
Permalink
Yes, and therefore the nbspinx `reader` component must care for handling
the various cases (complete Jupyter notebook vs. source text without
wrapper) accordingly.

No, nbsphinx does not provide a reader. In Sphinx translation,
sphinx-core provides it.
https://github.com/sphinx-doc/sphinx/blob/master/sphinx/transforms/i18n.py#L39-L65

So we need to enhance this to switch parsers with the format of the fragment.
Post by Komiya Takeshi
Hi Gunter,
As Sphinx uses Docutils for rST processing, the documentation in
http://docutils.sourceforge.net/docs/ may help.
Indeed Sphinx uses docutils' parser for its source parsers. So its
document is useful for developers.
But its responsibility is not same between docutils and Sphinx.
I meant no document describes it.
Yes, currently developers need to read both, Sphinx and Docutils
documentation and combine them to get the full picture.
Post by Komiya Takeshi
In Sphinx, source parsers should parse both whole of document and a
fragment of it to realize translation.
Usually they are same process for almost of source parsers.
The division of duties in Docutils means that the `parser` component is
ignorant about the source of the string it parses.
It is the duty of the `reader` component to extract the to-be-translated
rST source out of the context (a complete rST document, an rST document
fraction, doctrings of a Python file, multiple documents, ...) and hand
it as a string to the `parser`.
Post by Komiya Takeshi
But, in this case, nbsphinx should parse Jupyter notebook (JSON doc)
and a paragraph written in reST or markdown.
Because Jupyter notebook is a kind of container.
Yes, and therefore the nbspinx `reader` component must care for handling
the various cases (complete Jupyter notebook vs. source text without
wrapper) accordingly.
Depending on the source format, either the `rST parser` or the `markdown
parser` should be called to parse the source string.
The translation then works on the "document tree" returned by the parser.
Günter
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...