Documentation changes for replace and callbacks

Author: bruderstein

docs/source/intro.rst

@@ -70,7 +70,7 @@ as both sides are talking the same language.  However, there are a few things to
 what you want to do.  One important point is to make sure your script is saved in the same encoding as your target file(s) - this helps unicode strings 
 be interpreted the same way. 
 
-If you need to work with the string (for instance chance the case), you need to convert the string to a Python Unicode string.  To convert the string
+If you need to work with the string (for instance change the case), you need to convert the string to a Python Unicode string.  To convert the string
 append ``.decode('utf8')`` to the string. Obviously if your string is in a different format, use the name of the correct encoding.
 
 To put text back to Scintilla (so editor.something()), use .encode('utf8') from a unicode string.
@@ -201,16 +201,22 @@ To unregister a callback for a particular function, for particular events (perha
 The Callback smallprint
 -----------------------
 
-Due to the nature of Scintilla events, they are processed internally slightly differently to Notepad++ events.
+Due to the nature of Scintilla events, they are by default processed internally slightly differently to Notepad++ events.
 Notepad++ events are always processed *sychronously*, i.e. your event handler finishes before Python Script lets 
-Notepad++ continue.  Scintilla events are placed in a queue, and your event handlers process the queue (this happens
-automatically, you don't need to do anything different) - the only difference is that if you have a lot of callbacks registered,
-you might receive the event some time after it has actually occurred.  In normal circumstances the time delay is so small it
-doesn't matter, but you may need to be aware of it if you're doing something time-sensitive.  
-
-This does mean that cancelling the SCN_AUTOCSELECTION notification is not possible (as the SCI_AUTOCCANCEL message must be sent 
-before the notification returns).  I'm open to suggestions for this, but processing the notifications sychronously would mean 
-(slightly) reduced performance of "normal" code, added complication and more risk of obscure threading bugs.
+Notepad++ continue.  Scintilla events are placed in a queue, and your event handlers are called as the queue is *asynchronously* processed
+- i.e. the event completes before your event handler is complete (or potentially before your event handler is even called).
+
+The only visible difference is that if you have a lot of callbacks registered, or your callbacks perform a lot of work, you might receive
+the event some time after it has actually occurred.  In normal circumstances the time delay is so small it doesn't matter, but you may 
+need to be aware of it if you're doing something time-sensitive.
+
+ As of version 1.0, you can use :method:`Editor.callbackSync` to add a synchronous callback. This allows you to perform time-sensitive 
+ operations in an event handler. It also allows for calling :method:`Editor.autoCCancel` in a ``SCINTILLANOTIFICATION.AUTOCSELECTION`` 
+ notification to cancel the auto-complete.  Note that there are certain calls which cannot be made in a synchronous callback - 
+ :method:`Editor.findText`, :method:`Editor.searchInTarget` and :method:`Editor.setDocPointer` are notable examples.  :method:`Notepad.createScintilla`
+ and :method:`Notepad.destroyScintilla` are other examples in the ``Notepad`` object - note that this only applies to Scintilla (``Editor``) callbacks,
+ ``Notepad`` callbacks can perform any operation.
+
 
 
 .. _Python: http://www.python.org/

docs/source/scintilla.rst

@@ -4226,95 +4226,202 @@ Helper Methods
 
    Clears the callback for the given callback function for the list of events
 
+.. method:: Editor.callback(function, eventsList)
 
-.. method:: Editor.replace(search, replace)
+   Adds a handler for an ``Editor`` (Scintilla) event. The events list is a list of events to respond to, from the :class:`SCINTILLANOTIFICATION` enum.
+   Documentation on notifications from Scintilla can be found here: http://www.scintilla.org/ScintillaDoc.html#Notifications
 
-   Simple search and replace. 
+   For a simple example, here's a script that automatically saves the document as soon as a change is made::
 
+     def saveCurrentDoc(args):
+         notepad.save()
+   
+     editor.callback(saveCurrentDoc, [SCINTILLANOTIFICATION.SAVEPOINTLEFT])
 
-.. method:: Editor.rereplace(regex, replace)
+   This script is not really sensible in real life, as for large files, it will take some time to save the file, and it will be saved after every key press.
 
-   Simple regular expression search and replace (using Notepad++/Scintilla regular expressions).
-   
+   Note that ``Editor`` callbacks are processed *asynchronously* by default. What this means in practice is that your event handler function 
+   (saveCurrentDoc in the previous example) is called just after the event has fired.  If the callback handler is slow, and the callbacks occur quickly, you
+   could get "behind".  Callbacks are placed in a queue and processed in the order they arrived.  If you need to do something before letting the user continue, you 
+   can use :method:`Editor.callbackSync`, which adds a synchronous callback.
 
-.. method:: Editor.pyreplace(search, replace[, count[, flags[, startLine[, endLine]]]])
 
-   Python regular expression search and replace. Full support for Python regular expressions.  
-   Works line-by-line, so does not require significant memory overhead, however multiline regular expressions won't work (see pymlreplace_).  
+.. method:: Editor.callbackSync(function, eventsList)
+
+   Adds a *synchronous* handler for an ``Editor`` (Scintilla) event. The events list is a list of events to respond to, from the :class:`SCINTILLANOTIFICATION` enum.
+
+   What this means is that the handler function is called, and must complete, before control is returned to the user.  If you perform a slow operation in your handler
+   function, this will have an effect on the speed of Notepad++ for the user (i.e. Notepad++ may appear to have "locked up", whilst your event processes). 
+
+   Synchronous callbacks are mostly used for calling :method:`Editor.autoCCancel` in response to :class:`SCINTILLANOTIFICATION.AUTOCSELECTION`, but could be used for 
+   anything where the timing of the handler function is critical.
+
+
+.. method:: Editor.replace(search, replace[, flags[, startPosition[, endPosition[, maxCount]]]])
+
+   See :method:`Editor.rereplace`, as this method is identical, with the exception that the search string is treated literally, 
+   and not as a regular expression.
+
+   If you use a function as the replace argument, the function will still receive a ``re.MatchObject`` like object as the parameter,
+   ``group(0)`` will therefore always contain the string searched for (possibly in a different case if ``re.IGNORECASE`` was passed in the flags)
+
+   For example::
+
    
+   counter = 0
+
+   def get_counter(m):
+      global counter
+      counter += 1
+      return 'C' + str(counter)
+
+   editor.replace('(x)', get_counter, re.IGNORECASE)
+
+   # Replacing:
+   #
+   #     This (x) is some (X) text.  The bracketed X's will (x) get numerical replacements
+   #
+   # results in
+   #
+   #     This C1 is some C2 text.  The bracketed X's will C3 get numerical replacements
+
+
+
+
+.. method:: Editor.rereplace(search, replace[, flags[, startPosition[, endPosition[, maxCount]]]])
+
+   The main search and replace method, using regular expressions.  The regular expression syntax in use is 
+   that from Notepad++, which is actually the `Boost::Regex <http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/index.html>` 
+   implementation (specifically the Perl regular expression syntax). 
+
+
    ``flags`` are from the re module (e.g. ``re.IGNORECASE``), so ``import re`` if you use the flags.  
-   A maximum of ``count`` replacements are made, if zero or not given, then all replacements are made.
-   
-   ``startLine`` and ``endLine`` are optional, and if provided refer the first and last (zero indexed)
-   lines to perform the replace on.
 
-   *Note that as ``$`` only matches ``\n``, for Windows line ending files (i.e. ``\r\n``) if you want to match
-   the end of the line, you need to use ``\r$`` - see note about ``Editor.INCLUDELINEENDINGS``*
-   
-   As each line is searched individually, if you want to remove the line breaks, for instance to join lines together,
-   you need to add a flag ``Editor.INCLUDELINEENDINGS``. This includes the line endings in the search string, and also 
-   in the replacement.
-   
-   To just add a star (*) to the end of every line, you'd just use::
+   The ``re.MULTILINE`` flag is automatically set, so ``^`` matches the start of each line of the document, and
+   ``$`` the end of each line.  If you want to ``^`` and ``$`` to match the start and end of the whole document,
+   you can override the behaviour by adding in the ``editor.WHOLEDOC`` flag.  
+
+   Note that line endings are now handled automatically.
+
+   ``search`` can be a string, a unicode string, or an object. An object will be converted to a string using it's __str__ method.
+   For a unicode string, the current document encoding is checked, and an attempt is made at a conversion.  If the conversion cannot be 
+   successfully performed, an error occurs. When a standard Python string is used, no conversion takes place. If you need to replace 
+   strings in documents in both UTF-8 and ANSI (or other single byte encodings), then it's best to pass unicode strings.
+
+   ``replace`` follows the same conversion rules as ``search``.  However, you can also pass a function or lambda expression 
+   as the ``replace`` parameter.  This function receives a single parameter, which is an object resembling a re.MatchObject instance.
+   It only resembles an re.MatchObject because it doesn't support all the methods. Specifically, ``groupdict()``, ``pos``, ``endpos``, ``re`` and ``string``
+   methods and properties are not supported.  ``expand()``, ``group()`` and ``groups()`` (for example) all work identically.  The function should 
+   return the string to use as the replacement.
+
+   A simple function replacement::
+
+     def add_1(m):
+         return 'Y' + str(number(m.group(1)) + 1)
+
+     # replace X followed by numbers by an incremented number
+     # e.g.   X56 X39 X999 
+     #          becomes 
+     #        Y57 Y40 Y1000
+
+     editor.rereplace('X([0-9]+)', add_1);
+
+   ``startPosition`` is the binary position to start the search.  Use :method:`Editor.positionFromLine` 
+   to get the binary position from the (zero indexed) line number.
+
+   ``endPosition`` is the binary position to end the search. Use :method:`Editor.positionFromLine` 
+   to get the binary position from the (zero indexed) line number.
+
+   A maximum of ``count`` replacements are made, if zero or None, then all replacements are made.
 
-		editor.pyreplace("$", "*")
 
-   However, to join lines together, and put a pipe character (|) in between each one::
+   An small point to note, is that the replacements are first searched, and then all replacements are made.
+   This is done for performance and reliability reasons.  Generally this will have no side effects, however there
+   may be cases where it makes a difference. (Author's note: If you have such a case, please post a note on the forums
+   such that it can be added to the documentation, or corrected).
+
+.. method:: Editor.research(search, matchFunction[, flags[, startPosition[, endPosition[, maxCount]]]])
+
+   The main search method, using regular expressions.  The regular expression syntax in use is 
+   that from Notepad++, which is actually the `Boost::Regex <http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/index.html>` 
+   implementation (specifically the Perl regular expression syntax). 
+
+   ``flags`` are from the re module (e.g. ``re.IGNORECASE``), so ``import re`` if you use the flags.  
 
-        editor.pyreplace(r"\r$\n", "|", 0, Editor.INCLUDELINEENDINGS)
-		
-   Would result in::
+   The ``re.MULTILINE`` flag is automatically set, so ``^`` matches the start of each line of the document, and
+   ``$`` the end of each line.  If you want to ``^`` and ``$`` to match the start and end of the whole document,
+   you can override the behaviour by adding in the ``editor.WHOLEDOC`` flag.  
+
+   Note that line endings are now handled automatically.
+
+   ``search`` can be a string, a unicode string, or an object. An object will be converted to a string using it's __str__ method.
+   For a unicode string, the current document encoding is checked, and an attempt is made at a conversion.  If the conversion cannot be 
+   successfully performed, an error occurs. When a standard Python string is used, no conversion takes place. If you need to replace 
+   strings in documents in both UTF-8 and ANSI (or other single byte encodings), then it's best to pass unicode strings.
+
+   ``matchFunction`` is a function that gets callled with each match.  This function receives a single parameter, which is an object resembling a re.MatchObject instance.
+   It only resembles an re.MatchObject because it doesn't support all the methods. Specifically, ``groupdict()``, ``pos``, ``endpos``, ``re`` and ``string``
+   methods and properties are not supported.  ``expand()``, ``group()`` and ``groups()`` (for example) all work identically.  The function should 
+   return the string to use as the replacement.
+
+   A simple function replacement::
+
+     matches = []
+     def match_found(m):
+         # append the match (start, end) positions to the matches array
+         matches.append(m.span(0))
+
+     # find X followed by numbers
+     # e.g.   X56 X39 X999 
+     
+     editor.research('X([0-9]+)', match_found)
+
+   You can do the same thing with a lambda expression::
+
+     matches = []
+
+     editor.research('X([0-9]+)', lambda m: matches.append(m.span(0)))
+
+
+
+
+   ``startPosition`` is the binary position to start the search.  Use :method:`Editor.positionFromLine` 
+   to get the binary position from the (zero indexed) line number.
+
+   ``endPosition`` is the binary position to end the search. Use :method:`Editor.positionFromLine` 
+   to get the binary position from the (zero indexed) line number.
+
+   If ``maxCount`` is not zero or None, then the search stops as soon as ``maxCount`` matches have been found.
+
 
-		this is line 1
-		this is line 2
-		this is line 3
-		
-   Becoming::
+.. method:: Editor.pyreplace(search, replace[, count[, flags[, startLine[, endLine]]]])
+
+   This method has been removed from version 1.0. It was last present in version 0.9.2.0
+
+   You should use :method:`Editor.rereplace`.
+
 
-		this is line 1|this is line 2|this is line 3
 
 
 .. _pymlreplace:
 .. method:: Editor.pymlreplace(search, replace[, count[, flags[, startPosition[, endPosition]]]])
 
-   Python Multiline regular expression search and replace - works for multiline regular expressions, 
-   but makes at least 2 copies of the entire document, so is unsuitable for large documents. 
-   Note that re.MULTILINE is specified in the flags automatically.  
-   
-   ``flags`` are from the re module (e.g. ``re.IGNORECASE``), so ``import re`` if you use the flags.  
-   A maximum of ``count`` replacements are made, if zero or not given, then all replacements are made.
-   
-   ``startPosition`` and ``endPosition`` are optional, if provided they refer to the section of text, 
-   in offset bytes, to perform the search and replace in.
-   
-   *Note that as ``$`` only matches ``\n``, for Windows line ending files (i.e. ``\r\n``) if you want to match
-   the end of the line, you need to use ``\r$``*
+   This method has been removed from version 1.0. It was last present in version 0.9.2.0
 
+   You should use :method:`Editor.rereplace`.
 
+   
 .. method:: Editor.pysearch(expression, function[, flags[, startLine[, endLine]]])
 
-   Python regular expression search, calling a function for each match found.  
-   The function gets called with the (zero indexed) line number, and the match object. 
-   
-   ``startLine`` and ``endLine`` are optional, and if provided refer the first and last (zero indexed)
-   lines to perform the replace on.
-   
-   *Note that the match object start and end refer to the positions **within the line** and not the entire document.
-   Use :ref:`editor.positionFromLine` to find the location in the document*
+   This method has been removed from version 1.0. It was last present in version 0.9.2.0
 
+   You should use :method:`Editor.research`.
 
-.. method:: Editor.pymlsearch(expression, function[, flags[, startPosition[, endPosition]]])
 
-   Python multiline regular expression search, calling a function for each match found.  
-   The function gets called with the (zero indexed) line number, and the match object. 
-   
-   Note that this runs the search on the entire text, and therefore makes at least 2 copies of the entire document, 
-   therefore it may not be suitable for large documents.
+.. method:: Editor.pymlsearch(expression, function[, flags[, startPosition[, endPosition]]])
 
-   ``startPosition`` and ``endPosition`` are optional, if provided they refer to the section of text, 
-   in offset bytes, to perform the search in.
-   
-   *Note that the match object start and end refer to the positions within the entire document, unlike :ref:`editor.pysearch`*
+   This method has been removed from version 1.0. It was last present in version 0.9.2.0
 
+   You should use :method:`Editor.research`.