Importing a MultiMarkdown file into Scrivener with Import and Split

en
enricoscarpella
Posts: 3
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Fri Sep 14, 2018 10:14 pm Post

Dear all,

I am very new to Scrivener, so please go easy on me. I wrote a manuscript in Multimarkdown Composer, and I have imported it into Scrivener by:

File > Import > Import and Split.
- [x] Split and structure using MD headings in the text
- [x] Convert MD

The first few lines of the original .md file are:

# Title
##### Authors
###### Affiliations

After importing, however, the Authors document is a direct descendant of the Title document, and the Affiliations document is a direct descendant of the Authors document. Not surprisingly, therefore, when I compile the manuscript (Pandoc -> Microsoft Word (.docx), for example), the Authors are styled as Header 2 and the Affiliations as Header 3. For now, I have solved the problem inelegantly by adding a hierarchical series of three untitled documents between the Title document and the Authors document, but I am wondering if there is a way to automate this. Thank you for your consideration.

Sincerely,
Enrico

User avatar
AmberV
Posts: 22347
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Sat Sep 15, 2018 10:20 am Post

I don’t understand what it means to have gaps in the hierarchy like that. How can there be a level four heading all by itself? The only reason for there to be a level four is because there is a level three above it. :) Well no matter, outliners like Scrivener anyway do not operate in such a freeform manner, so the gaps in your header structure are being handled as best it can interpret the request.

As an aside, I’d also question whether you should use the Convert Markdown checkbox when importing. Since you are compiling via Pandoc to DOCX, you would be wanting to use Scrivener to compose Markdown in most cases—i.e. you’d be writing in Scrivener just like you did in Composer. Converting the Markdown on import will strip most of the markup out and be a more suitable option if you intend to use Scrivener’s own .docx generator—which is a completely different way of using Scrivener.

Of course the simplest answer is to simply go on using Markdown in Scrivener and maintain the heading structure yourself, since it won’t be conforming to the outline structure. In that case you would set up the compiler to not insert any headings at all, since they would already be in the source text.

But there are also ways to force the issue with the number of hashes used by the compiler. The default is “by level”, but you can create Section Layouts that use a specific number of hashes no matter where they occur within the outline:

  1. In your project, use the Project ▸ Project Settings... menu command, and click on the “Section Types” tab. You want to create at least a couple of new Types for these specific headings. The details of how to do that are documented in Appendix C.2.2, Section Types Tab. If this is all new stuff to you, it might be worthwhile to start at the top of C.2, and maybe read §7.6, Section Types as well, for basic theory.

    I create two types, “Author Page” and “Affiliation Page”. That’s all we need to do here, so click OK to save the settings.
  2. You now want to identify the appropriate documents in the binder as being assigned to these types. If there are many of them, the easiest way will be to Cmd-click to select all of the “Authors” sections, right-click on them, and use the Section Types submenu to pick the “Author Page” type. If there is only one you could do the same thing—but you can also set Types on a one-by-one basis in the Inspector’s metadata pane. Section Type assignments in general are documented in §7.6.1, Applying Section Types Manually.
  3. Now that you have your binder outline tagged correctly, you need to set up your compile settings to do things with these Types. Open File ▸ Compile.... I’m starting with the “Basic MultiMarkdown” compile Format, so right-click on that in the left sidebar and duplicate and edit it.
  4. Section Layouts are what we use to describe how a thing is formatted when it compiles. With Markdown, “formatting” is structural declaration rather than fonts and such of course, so we specifically mean here: how many hashes should these sections have. You can read about creating new Layouts in §24.2.1, Section Layout List.

    I’ll create two different layouts called “Text with Level 4 Heading” and “Text with Level 5 Heading”. I use the “Text Section with Heading” as a starting point, since that is the closest Layout to what we want. Specifically it already has the “Title” and “Text” checkboxes applied. If you want something different, start from either of the other two Layouts or just change the checkboxes—this basic format is really basic. The only difference between the Layouts in this list are in fact the checkboxes.
  5. For the “Text with Level 4 Heading” Layout, click on the “Title Options” tab in the lower half of the pane, and use the Number of hashes setting to select “4”. Do the same for the Level 5 Layout, only choosing “5” of course. If you click back to the “Formatting” tab, you’ll see the number of hashmarks you desire. Click the Save button.
  6. Lastly, you need to tell the compiler that you want to use these new Layouts for the Section Types you created earlier. Click the Assign Section Layouts... button along the bottom of the preview column.
  7. Select “Author Page”, and choose “Text with Level 4 Heading”. For “Affiliation Page”, select “Text with Level 5 Heading”. Click OK to save your assignments.
  8. Note while you’re here, in the Contents column on the right, you can see an overview of the outline and its Section Type assignments. This is another way in which you could assign Types to items, and it is useful if you noticed that you missed a spot earlier.
  9. Click Compile, and now you should be at the point you want. :)
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

en
enricoscarpella
Posts: 3
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Sat Sep 15, 2018 4:20 pm Post

Dear AmberV,

Thank you very much for the prompt, exhaustive and helpful reply; I will look into those project and compile settings to test the solution you kindly suggested.

As to your comment:

I don’t understand what it means to have gaps in the hierarchy like that. How can there be a level four heading all by itself? The only reason for there to be a level four is because there is a level three above it.


It is simply because in this way, when I export to .fodt from Multimarkdown Composer or to .fodt, .docx or .pdf through Marked2 — either with the built-in Multimarkdown processor or with Pandoc as custom processor — the text:

# Title

##### Authors

###### Affiliations

is automatically styled as per Journals' Authors Instructions. So here the header level does not reflect the hierarchical structure of the document, and I take advantage of the fact that in the rest of the manuscript — where instead the header level does reflect the hierarchical structure of the document — I never use Header 5 and 6, so I can reserve them to style authors and affiliations as required by journals.

And as to your other comment:

As an aside, I’d also question whether you should use the Convert Markdown checkbox when importing. Since you are compiling via Pandoc to DOCX, you would be wanting to use Scrivener to compose Markdown in most cases—i.e. you’d be writing in Scrivener just like you did in Composer.


That is of course a possibility; my understanding, however, is that a major difference between writing in Multimarkdown in Scrivener and writing in Multimarkdown in Multimarkdown Composer is that the latter, but not the former, has syntax highlighting — which I can style however I prefer in Multimarkdown Composer. That is why I had not considered the possibility of writing in Multimarkdown directly in Scrivener. But perhaps I am mistaken and there are ways to highlight syntax in Scrivener by creating new styles…

Again, thank you very much for taking the time to help me out.

Best regards,
Enrico

User avatar
AmberV
Posts: 22347
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Sun Sep 16, 2018 10:14 am Post

Ah okay! Thanks for explaining how the heading styles are used.

Is the journal actually asking for h4 and h5 headings, or is this just a convenient mapping to get the text into styles, so they can be easily switched to the named styles the journal requires?

If it is the latter, you might look into Pandoc’s ability to assign arbitrary style names to ranges of text. Search for the heading Custom Styles in Docx Output in the Pandoc documentation. Here is an example he provides:

Code: Select all

[Get out]{custom-style="Emphatically"}, he said.


There is also a format that can be used for longer blocks of text as well. If you use a Pandoc reference.docx file you can add the “Emphatically” style to that document and it will be styled as you require when compiling from Scrivener.

As for how to actually do that in Scrivener (other than just typing it in of course), I’d use Styles for that. When you edit a compile Format, go into the Styles option pane, where you can add a reference to the style name and then have the compile insert that code around the text. The prefix could be “[”, and the suffix all the rest after the word “out”.

Well, even if you don’t need it for that specific purpose, hopefully you’ll find the technique of use for many other things. I use that approach for all manner of stuff that MMD and Pandoc themselves do not have native support for in their syntax, or wherever I wish to to delegate specific semantics to the compiler, leaving the semantics in the editor more generalised. “Emphatically” can be a Word style in some cases, or it might just be italics in HTML. While writing I am not bothered by that distinction—and that is one of the reasons why I consider Scrivener to be a very interesting platform for Markdown composition.

That is why I had not considered the possibility of writing in Multimarkdown directly in Scrivener. But perhaps I am mistaken and there are ways to highlight syntax in Scrivener by creating new styles…


To preface, I come from the school of thought where one of Markdown’s big achievements is that you can create sophisticed documents with nothing more than a WordStar level word processor or text editor. Indeed, my AlphaSmart NEO is little more than that, and its lack of styles, syntax highlighting or even basics like italics does not stop me from drafting complex documents. My approach to Markdown in Scrivener has always been from that perspective: that I can type Markdown into whatever window I want. For me having that window quasi-format the text in real time is interesting, but not necessary. I’m more interested in having that window hooked up into an extensive framework for generating, managing and coordinating large amounts of information.

The editing environment itself is quite different from a plain-text based editor, this is true, and how to make use of it is largely a matter of taste. I go over what I feel to be the three main approaches to using Scrivener+Markdown, in §21.5, Markdown and Scrivener. All three methods have good workflows in my opinion. Consider that with a pure plain-text approach you are not stuck writing in Scrivener. If you prefer aided typing and syntax highlight, then you can copy a chunk out of the binder into Composer, draft away, then copy and paste back in (and one neat trick with Composer is that RTF copy method which makes the copy look just as nice in Scrivener. Scrivener becomes your repository and final assembler—it is where you direct the writing, but maybe not so much for doing the initial typing itself. In such a case, one my prefer to design the writing environment around a plain-text feel.

Whether I use a more purist approach or a blend of converters and markup depends a lot on the project. For this one here, where I store communications relating to Scrivener, I just type in all the markup myself, like I would be if I used Composer to write, and use no features because I never compile out it. I use a macro that converts the buffer to BBCode and pastes it into my browser, or I copy and paste into our help desk which takes Markdown direct. For me, in this project, using styles and such as functional features would greatly impede the usefulness of the workflow. But in other projects, features like styles, figure and footnote management and so forth are huge time savers.

If you want to see a practical example of how I tend to work, download the user manual project. That’s a fully capable Scrivener+MMD+LaTeX project. You’ll see I use a mix of styles to generate more complex stuff (like the above example, but for LaTeX), and I use some rich text stuff like lists and images, but I also use a lot of Markdown as well while typing. To my eye, I don’t think the content would benefit much from syntax highlighting. But then again I do prefer minimalist themes in the editors that offer it. I like “Empty” in Composer for example. :)
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

en
enricoscarpella
Posts: 3
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Fri Sep 28, 2018 3:42 am Post

Dear AmberV,

Thank you very much once again for taking the time to provide such an exhaustive and informative reply.

My apologies for my belated reply: I was away on an assignment; I have just got back and have now started to catch up.

Is the journal actually asking for h4 and h5 headings, or is this just a convenient mapping to get the text into styles, so they can be easily switched to the named styles the journal requires?


It is the latter, so your suggestion (i.e. exploring Pandoc's .docx Custom Styles) is right on the mark. In fact, I have a series of .docx files of older manuscripts formatted according to the guidelines of the journals in which they got published that I use as custom templates for Pandoc. In this way, if a manuscript is unfortunately rejected by a journal, reformatting that manuscript for submission to a different journal is only a matter of seconds. But I had not considered the possibility of assigning arbitrary style names, which is a very helpful suggestion: thank you very much! And thank you very much also for suggesting how to insert the code for those customs styles during compiling!

For me having that window quasi-format the text in real time is interesting, but not necessary.


You are right, of course; however, in my field it is very common to have to write long series of symbols whose combination, order, separators and formatting convey different meaning. So to avoid mistakes, I prefer to style all that formatting in MMDC with my custom theme. Further, my students and I make extensive use of CriticMarkup to collaborate on text, and I find it hard to navigate through comments on comments on comments as I put together the final version of a manuscript. The CM support in MMDC is excellent, but I still like to enhance it by styling its syntax in my custom theme.

If you prefer aided typing and syntax highlight, then you can copy a chunk out of the binder into Composer, draft away, then copy and paste back in (and one neat trick with Composer is that RTF copy method which makes the copy look just as nice in Scrivener. Scrivener becomes your repository and final assembler—it is where you direct the writing, but maybe not so much for doing the initial typing itself. In such a case, one my prefer to design the writing environment around a plain-text feel.


Thank you very much for all this: first of all, for the RTF-copy trick; second, for an enlightening perspective on Scrivener as not necessarily the place where the writing takes place but where the writing is directed; and third, for the link to your post on how to emulate plain text writing in Scrivener.

If you want to see a practical example of how I tend to work, download the user manual project.


Thank you very much: that is indeed very informative; however, I would also be interested in how the work in progress looks like in addition to how its final product does. For example, I wonder whether and how you use revisions and annotations, and how you collaborate on text in Scrivener. I understand I am asking a lot and that this might not be of sufficient interest to the Forum, so please feel free — as always, of course! — to ignore this. Either way, you have already helped me a lot; for this, thank you very much again.

Best regards,
Enrico

User avatar
AmberV
Posts: 22347
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Fri Sep 28, 2018 12:21 pm Post

No worries, and you’re quite welcome! The nice thing about forum conversations is they can extend into time as long as they need to. :)

Further, my students and I make extensive use of CriticMarkup to collaborate on text, and I find it hard to navigate through comments on comments on comments as I put together the final version of a manuscript. The CM support in MMDC is excellent, but I still like to enhance it by styling its syntax in my custom theme.


That in particular is something I’d like to see improved on our end some day. Given the simplicity of CM as a syntax, it shouldn’t be too difficult to have these constructs turned into inline annotations and inspector comments, based on usage. Where it gets trickier are the add/del/highlight codes. Deletions could map to overstrike naturally, and highlights also are a thing, but personally I’d rather have all three of those map to styled text, where in the default MMD outputs there are styles for exporting the text to CM. But that gets more complicated in terms of input settings—that’s the part I haven’t quite figured out an elegant solution for yet.

Thank you very much: that is indeed very informative; however, I would also be interested in how the work in progress looks like in addition to how its final product does.


That’s probably a long post in and of itself. :) Fortunately I’ve written a bit on the topic of how I use inline annotations. I use a system of code words to prefix them, which in turn makes it simple to gather documents that contain such markers. For example if I put “TODO//” into an annotation in a document, that document automatically gets added to my “To Do” search collection, and now I have an interleaved to do list that I can work through before releasing a revision.

As for revision markings—I used to use the feature a lot more than I do now. These days I tend to rely on Snapshots, and their comparison feature which largely does what revision marking does but without the extra effort of making those markings myself. I’ve drifted to that approach as the Snapshot feature became more powerful over the years. A few things that make it a suitable retroactive change tracking system:

  • The General: Saving preference pane option, Take snapshots of changed text documents on manual save. I also have my backup preferences set to back up the project when I manually save. So for me, hitting ⌘S is a full stop, or a milestone point. Snapshots are built up automatically, and their nature of being built only at milestone points in the process means they tend to represent meaningful change notes.
  • Better and easier ways to view comparisons alongside the main work. In the past you could Option-drag a snapshot from the inspector list into an editor to view a change log between it and the current state of the text. But now I can just right-click and load the comparison (again with Option held down) into the other editor, or even a copyholder. It’s a small adjustment, but sometimes that’s all it takes.
  • The Documents ▸ Snapshot ▸ Snapshots Manager tool, on the other end of the process, makes it easy to clean up all of these old automatic snapshots.

That’s good for when I’m tracking my own work. With collaboration I think a more “approve/disapprove” based approach is better, and for that revision markings are a better tool. Of course that assumes everyone is using Scrivener and working from a common source. If that is the case, check out §5.3.2, Merging Projects in the user manual, and specifically the notes on using this capability for collaboration, starting on pg. 77. For an all-Scrivener work group, this is going to be a very effective way for multiple people to work in parallel on the same material. And if they are using comments and revision marking it should all be very clear for the person doing the final integration of everyone’s notes and contributions.

For collaborating with people not using Scrivener; I don’t have as much experience with that side of things. I’m lucky in that everyone I work with uses Scrivener. ;)
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles