Wincent Colaiuta [Wed, 23 Jun 2010 16:12:20 +0000 (18:12 +0200)]
Format all "pre" blocks for YARD
This means marking all non-Ruby "pre" blocks using a "!!!wikitext" or
"!!!html" marker.
At the moment there is no syntax highlighting for wikitext or HTML, so
those tags are effectively equivalent to "!!!plain", but the important
thing is that we stop YARD from inappropriately applying Ruby syntax
highlighting rules to non-ruby code, and if syntax highlighting for the
other languages is ever added, we will be ready for it.
Wincent Colaiuta [Wed, 23 Jun 2010 15:34:47 +0000 (17:34 +0200)]
Bump YARD requirement to 0.5.8
From the official announcement:
Yesterday I released YARD 0.5.7. Unfortunately a few important bug fixes
were omitted from the release because they were lost in the 0.6-master
branch and I did not merge them back. That was a total screwup on my
part. Hopefully this doesn't happen again (some extra specs were added
to make sure this issue stays fixed).
Wincent Colaiuta [Sun, 13 Jun 2010 17:37:24 +0000 (19:37 +0200)]
DRY up gemspec
Rather than repeating the gem names (4 of them), repeat the
"respond_to?" check (1 check) each time through the loop.
Speed not an issue, here, so just run the check every time
through the loop rather than sacrificing readability by
cluttering things up with a local variable.
Wincent Colaiuta [Sun, 13 Jun 2010 09:29:15 +0000 (11:29 +0200)]
Backend support for HTML5/XML output styles
HTML5 has two output syntaxes, HTML and XML. Under HTML5 the HTML output
syntax is likely to be predominant, given that the XML syntax can only
be used if it is served with an appropriate MIME type (ie.
application/xml), and the vast majority of deployed web servers are
configured to use a text/html MIME type.
So, this dictates that for our 2.0 release we are going to default to
emitting HTML syntax, but allow the user to override the default and
emit XML syntax if required.
In practice, the only place in the code base where this necessitates a
change is the part where we emit "img" tags. In the XML syntax, these
tags should be self-closing; ie:
<img src="foo.png" alt="Foo!" />
Whereas in the HTML syntax they should not (even if all user agents
accept them anyway); ie:
Wincent Colaiuta [Sat, 12 Jun 2010 23:01:35 +0000 (01:01 +0200)]
Run specs under Rails 3.0.0.beta4
Note how that when targetting Rails 3 release versions, we can't use the
version number as the app name as we did with Rails 2.
This is because Rails 3 tries to turn the name into a constant, and
constants can't start with a number. To work around this we take such
version numbers and prepend them with a "v" to create a test app name;
this in turn can be "constantized" into "V300Beta4" or similar.
Wincent Colaiuta [Sat, 12 Jun 2010 20:37:22 +0000 (22:37 +0200)]
Start refactoring Rails specs
The "Edge" Rails specs have been broken for a long time as we really
need a separate code path for these (too many changes in the underlying
machinery).
Lay the groundwork by renaming the existing working infrastructure as
a "rails2".
Early on in the application boot process. In our case, that effectively
means that the code in the C extension's Init_wikitext() function will
be evaluated.
So, in this commit, we replicate in C what we would have done in the
following manner if the Wikitext gem were just a pure Ruby project:
if Object.const_defined? :ActiveSupport
# we are running under Rails
require 'wikitext/nil_class'
require 'wikitext/string'
if ActiveSupport.respond_to? :on_load
# running under Rails 3
ActiveSupport.on_load :action_view do
require 'wikitext/rails_template_handler'
end
else
# running under Rails 2
require 'wikitext/rails_template_handler'
end
Note that because one of the code paths involves passing in a block, we
have to define a couple of ancilliary helper functions and use
rb_iterate() to achieve the same effect in C.
Wincent Colaiuta [Sat, 12 Jun 2010 16:08:10 +0000 (18:08 +0200)]
Split String#wikitext_preprocess into a separate file
This has always been a bit of a wart, the fact that what is essentially
an application-specific preprocessing pass was included in our
"wikitext/strings" file.
Now, if you want the behavior you have to explicitly require
'wikitext/preprocess'.
As a side effect of this change, the file now serves as a nice example
of how to implement a custom preprocessing phase if you need it.
Wincent Colaiuta [Fri, 11 Jun 2010 16:33:55 +0000 (18:33 +0200)]
Emit <code> rather than <tt> elements
The <tt> element has been dropped from HTML5 and the recommendation is
to instead use more "semantic" elements such as <code>, <samp> and
<kbd>. As one of the major goals for the 2.0 release is to emit valid
HTML5, the <tt> element has to go.
Given that we are working with wikitext markup and the focus is on
simplicity, we won't be adding new syntax to enable the user to control
which of the three "semantic" tag alternatives is emitted.
Instead, we maintain the existing input markup options (that is,
backticks and <tt> elements will continue to be recognized) to maintain
compatibility with existing bodies of wikitext markup, and we choose
one of the three possible elements as a replacement for <tt>.
In the end, we're going with <code> because it seems to be the most
frequently used in the terrain where the wikitext module is used (online
wikis). A customization option to override this choice could be added in
the future, but I honestly don't expect there to be any demand for it.
Likewise, explicit support for <samp> and <kbd> tags could be added, but
I doubt there'll ever be a call for it, as the whole point of working in
wikitext markup is to have an easy-to-use format and not have to write
HTML tags.
Wincent Colaiuta [Thu, 10 Jun 2010 18:41:05 +0000 (20:41 +0200)]
Run Rails specs against many more versions of Rails 2
Previously we only ran against Rails 2.2.2 (the latest release at the
time the specs were written) and Rails "Edge".
In this commit we extend that to run against all versions of Rails 2
put out since the release of 2.2.2. The only exception to this is
version 2.3.0, which is marked as pending because it throws a Rack
exception. I am not sure if the version of Rack bundled with 2.3.0 has
some kind of bug, or this is an interaction with the versions of Rack
that are on my system (1.0.1 and 1.1.0) and might be getting
inadvertently pulled into the tests.
Note that we don't bother running the specs against versions of Rails
older than 2.2.2, because that came out in November 2008, and there is
not much sense in retroactively extending compatibility even further
backwards in time.
Wincent Colaiuta [Wed, 9 Jun 2010 21:33:32 +0000 (23:33 +0200)]
Move to Rails 3 template UI
It seems that under Rails 3 the old "self.call" method no longer works
in the same fashion. Previously returning the string "template.source.w"
worked because Rails, evidently, ended up eval'ing it in a context where
"template" resolved to the desired object.
In Rails 3, however, this is no longer the case, and it ends up trying
to call a "template" method on the controller, failing.
Googling for different template handlers shows that some use "render"
(for example, liquid) but I am unable to get any variant of "render" to
work under Rails 3.
On the other hand, the three handlers included with Rails itself (ERB,
Builder, and RJS) all use the "compile" method, as does Haml.
It's hard to tell from the scant in-code documentation, but it looks
like "compile" might be "the" supported way to write template handlers
for now. The only reference I can find to this online is this thread
from back in 2008:
http://www.ruby-forum.com/topic/159810
In which Josh Peek says:
I'd really like deprecate the TemplateHandler#render all together. If
you think your handler can not be compiled, you always just compile in
a delegate call to your "noncompilable" handler. This leaves us with
only one method, compile.
So I am not sure if this is the "official" stance on the API, but in
light of the fact that I can't get anything else to work, I am going to
go with "compile" for now.
This unfortunately requires a somewhat ugly gsub call in order to escape
any single quotes that might be in the input, but it seems a necessary
evil. For example, a template with this wikitext markup:
* don't you like this list?
Will be "compiled" to:
'<ul>
<li>don\'t you like this list?<li>
</ul>
'
Which when evaluated and emitted to the output steam becomes:
<ul>
<li>don't you like this list?</li>
</ul>
Note that this commit makes things work with Rails 3 only, breaking
Rails 2 support. In a subsequent commit I plan to add in a conditional
code path that kicks in when running under Rails 2.
Wincent Colaiuta [Wed, 9 Jun 2010 19:33:58 +0000 (21:33 +0200)]
Mark String#w and String#to_wikitext output as HTML safe
This applies only when running under Rails 3, or when the "html_safe"
method is otherwise available.
As these methods are the most commonly used methods for translating
wikitext markup to HTML in Rails applications, this should provide a
fairly convenient upgrade path for developers moving from Rails 2 to
Rails 3.
These were originally intended to be run independently as Ruby scripts,
but in practice they are only run using the "spec" tool, so remove the
bang line and unset the executable bit.
Seeing as it seems that Rails 3 and Bundler no longer automatically
execute the "rails/init" file, users will now have to require
"wikitext/rails" if they want to get the functionality that was
previously activated from the "rails/init" file.
As mentioned in commit 4488c7b, in the library files we trust either the
packaging system (usually RubyGems) or the user to correctly set up the
LOAD_PATH so that all files will be found when included using simple
relative paths like "wikitext/version" and so on.
The specs and benchmarking scripts already explcitly set the LOAD_PATH
and use absolute paths where appropriate to ensure that the local
version is tested rather than some other version that might be installed
elsewhere on the system.
This commit addresses the last missing piece of the puzzle, the
"bin/wikitext" executable. That was also changed to trust the LOAD_PATH
and use shorter relative paths, and this will work for the installed
case because the packaging system (RubyGems) will actually set up a
wrapper in the user's PATH which sets up the LOAD_PATH appropraitely.
In local testing however, no such wrapper script is in place so trying
to run "bin/wikitext" will either fail (when no version is installed on
the system in th PATH) or will most likely use the wrong version (when
another version is found elsewhere).
So, for the purposes of local testing we here add a wrapper script that
achieves the same affect as the wrapper created by RubyGems during
installation: we set up a local LOAD_PATH and then fire off the real
executable.
Trust the packaging system (usually RubyGems) to set up the LOAD_PATH
appropriately, thus allowing us to use cleaner and simpler "require"
startements.
Note that the spec suite already explicitly sets up the LOAD_PATH so
that the version in the current working directory is tested and not any
other versions which may be installed elsewhere on the system.
The "bin/wikitext" executable, however, doesn't do any special setup so
it will use the installed version if it is found first in the load-path.
This isn't a problem for real installs because the packaging system (ie
RubyGems) sets up an appropriate wrapper executable to set up the
LOAD_PATH first. (For local testing, I still need to set up such a
wrapper.)
This is the reincarnation of commit 829011d, which fixed a subtle
problem with a missing file in the built gem.
We basically have to make sure that we've run the "make" target so that
all of the C files, including the Ragel-generated C file, is present in
the directory at the time "gem build" is called.
There is still nothing very interesting on the Wikitext product page at
wincent.com, but it is still a more permanent home than rubyforge.org is
likely to be in the long term.
Note that while there is some disagreement in the comments on that post
about whether this is a clever or correct thing to do, in the present
case we are merely extracting the hand-edited gemspec out into a
separate file, not "checking in a generated file into the repo".
The change seems justified because it buys us compatibility with a
Bundler feature at no real cost.
Likewise, the other main complaint raised on a different post on the
same blog (ie. that building from Git repos can run into complications
where you end up with two different gems with different code but the
same version number) doesn't apply here because I have always used a
sane version numbering scheme, bumping the version number after each
release, so a released gem will never have the same version as an
intermediate version built directly from the repo:
Wincent Colaiuta [Sun, 21 Mar 2010 17:32:57 +0000 (18:32 +0100)]
Fix builds on 64-bit Darwin platforms
Way back in ancient history when wikitext still used an ANTLR-generated
lexer, the evil commit 7a510102 added a horrible specific ARCHFLAGS
override to overcome the fact that ANTLR didn't produce Universal
binaries.
This worked fine on 32-bit machines, and the evil commit was allowed to
persist through the codebase for over two years, long after ANTLR was
ripped out and replaced with the vastly superior Ragel-generated lexer.
Today on testing on a 64-bit Darwin machine for the first time I
discovered the evil, and this commit expurgates it.
Wincent Colaiuta [Sun, 1 Nov 2009 13:26:55 +0000 (14:26 +0100)]
Teach wikitext tool to accept file arguments
If additional arguments are supplied they will be interpreted as
files to be processed in non-interactive mode (and any input on STDIN
will be ignored).
If input is from a pipe or output is to a pipe we enter non-interactive
mode and just process the input from STDIN and send it to STDOUT without
any prompts, looping, or colorization.
This is a partial implementation of the feature requested here:
https://wincent.com/issues/1399
(The remaining part is to also accept filename arguments as a means of
entering non-interactive mode.)
The "wikitext" command-line tool has been in the repository for quite a
while, but never included in the Gem distribution. Included by user
request; see:
Wincent Colaiuta [Thu, 6 Aug 2009 12:11:21 +0000 (14:11 +0200)]
Silence compiler warning in StringValue
Our use of "StringValue(j)", where "j" is of type "long" and the macro
wants a value of type "VALUE" ("unsigned long") produce a warning that
"pointer targets in passing argument 1 of ‘rb_string_value’ differ in
signedness".
Merely casting ("StringValue((VALUE)j)") doesn't work because
"StringValue" itself is a macro and produces this warning: "argument to
'&' not really an lvalue; this will be a hard error in the future".
So use a temporary variable to suppress the warning without having to
use a cast inside the macro.
Wincent Colaiuta [Wed, 27 May 2009 15:49:24 +0000 (17:49 +0200)]
Use fixed-width markup in release notes for class names etc
Make things more consistent with the rest of the documentation by using
fixed-width font markup for CSS class names and HTML tags in the release
notes (rather than using double quotes).
Wincent Colaiuta [Tue, 26 May 2009 20:28:18 +0000 (22:28 +0200)]
Update README for new PRE_START "lang" attribute
Document the use of the "lang" attribute in PRE_START tags. Note that
contrary to what is said in the commit message for 024870e, uppercase
letters are in fact allowed in the attribute value.
Wincent Colaiuta [Tue, 26 May 2009 20:03:31 +0000 (22:03 +0200)]
Handle optional "lang" attribute in PRE_START tags
In addition to the literal "<pre>" tag syntax accept a special syntax
which includes a "lang" attribute. This is used to markup a PRE_START
for syntax highlighting as a particular language; for example:
Note that language names must consist only of lower-case letters, and
"-syntax" is automatically appended, to prevent users from inserting
totally arbitrary CSS class names into the translated output. This means
that names like "ruby" and "c" will work, but that names like "obj-c" or
"ObjC" would have to be written as "objc".
In this implementation the required format for the PRE_START tag is very
strict; no excess whitespace is allowed within the tag. This has the
benefit that it is simple and fast, and doesn't require re-scanning the
token to see exactly where the value of the "lang" attribute begins and
ends.
Wincent Colaiuta [Thu, 21 May 2009 19:13:34 +0000 (21:13 +0200)]
Really change license header
90de8af corrected the license header in the generated C file rather than
the Ragel source (I need to add an "rl" filetype to Ack so that I won't
overlook this kind of thing in the future).
Wincent Colaiuta [Wed, 13 May 2009 21:09:44 +0000 (23:09 +0200)]
Swap README.rdoc and doc/README (symlink <--> file)
Looks like a top-level symlink won't get picked up by GitHub,
so swap things around: make the top-level item be the real
file and the item in the "doc" subdirectory can be a symlink
(RDoc will continue to generate the HTML for the README as
before).
Wincent Colaiuta [Wed, 13 May 2009 16:08:40 +0000 (18:08 +0200)]
Add README symlink at top level for GitHub mirror
GitHub will automatically display any README found at the top
level of the tree, so set up a symlink (we'll see if it works)
from doc/README to README.rdoc.
Wincent Colaiuta [Wed, 13 May 2009 09:04:16 +0000 (11:04 +0200)]
Add ary_includes2, ary_includes3 functions
A fairly common pattern in the codebase is successively calling
the ary_includes function 1, 2 or 3 times. In the worst case
scenario this incurs 3 function calls and three array traversals
via a for-loop in the function.
Add 2 and 3-argument variants of the function so that we can
replace these instances with a single function call and a single
array traversal.
As shown in the benchmarking notes, this bumps the ary_includes
function from 4th place in the profile (where it was the "low-hanging
fruit") and right off the list.
Wincent Colaiuta [Tue, 12 May 2009 22:44:49 +0000 (00:44 +0200)]
Bump version number for 1.7 release
Normally "minor" version bumps (eg 1.6 to 1.7) are
reserved for versions which introduce new features
and "tiny" version bumps (eg 1.6 to 1.6.1) are for
maintenance (bugfix) releases.
This release doesn't contain any new features but
I am nevertheless bumping the "minor" version
number because the release does involve a fairly
major rewrite of the internals, and as such is
less conservative than a normal "maintenance"
update.
Wincent Colaiuta [Tue, 12 May 2009 22:29:20 +0000 (00:29 +0200)]
Change internal function prefix from "_Wikitext_" to "wiki_"
Follow the pattern already established in str.c/str.h and
ary.c/ary.h, and use a short lowercase prefix with no
leading underscore for "internal" functions (that is,
functions that are for use within the C part of the extension
and are not exposed as Ruby methods).
This buys us some horizontal space and makes keeping things
in a reasonable number of columns a bit easier.
Externally exported methods still follow the old pattern of
Module_class_method (eg. Wikitext_parser_parse).
git.wincent.com / wikitext.git / log
