Some more comments after using emacs for a while:

;; Disable all the version control stuff         
;; Makes emacs load much faster inside git repos 

(setq vc-handled-backends nil)

;; ===== Enable mouse support ====
                                      
(require 'xt-mouse)                   
(xterm-mouse-mode)

;; ===== isearch+ =====         

(require 'isearch+)

Some emacs questions:

This Tuesday will mark the point where I will have spend as long using emacs as I did using vim. But already, I feel more competent with emacs. I won’t repeat what I said in my last post, but I just want to say that the ability to edit and write at the same time makes me way more productive. The fact that it uses keyboard shortcuts that I’m already used to probably helps a lot too. Even so, I’ve not used any kind of cheat sheet for emacs (since I never really found any that were any good), and yet I feel like I’ve memorized more key commands now than I ever did with vim, for which I did use a cheat sheet.

So I really don’t see myself going back to vim at this point.

I’m actually surprised. Virtually everyone I know who uses a command line editor uses vim. It’s definitely the more popular of the two. But having tried both, I can only speculate as to why. Vim has a much higher learning curve than emacs. Everybody grows up learning how to write text in editors like Microsoft Word, TextEdit, Notepad, etc., that all work fundamentally like emacs: if you type text, it enters the text. If you want to do advanced editing with the keyboard, you hold down some meta keys and type chorded keyboard shortcuts. The vim modal editing methodology is so different from this, that it surprises me that so many people go to the trouble of learning it (I mean, to the point that they are more efficient with it). I can see the benefit over GUI editors, which have nothing on either vim or emacs with regards to customization, or just the plain editing power that is really necessary for coding. My guesses why people use vim:

Maybe the vim users out there could comment why they use vim. Am I missing something? Or are your heads just wired differently from mine? And if you use emacs (or anything else), I’d love to hear from you too?

At any rate, I recommend that anyone who wants to give command line editors a chance do what I did: learn both vim and emacs. My blog posts should be enough to give you some good advice. I went cold-turkey, and I recommend that you do too, but only do it if you won’t have any important editing to do for a few weeks, as your editing rate will slow down a lot as you are learning for both editors. And even though I think I am going to stick with emacs, learning vim was still valuable. Unlike emacs, vi is part of the POSIX standard, so it’s included in pretty much every UNIX distribution. I’ll be glad when I find myself on a minimal command line and know how to use a decent text editor. And anyway, you can’t really know which one will be your way until you try them both. I really thought I would end up using vim, as it was so popular among all the people I know who use command line editors. But I guess there is only One True Editor.

EDIT: I found out how to make emacs really fast. The key is to run one process of emacs in daemon mode, and have the rest connect to that. Then you only have to wait for the startup once (per computer session). To do it, just set your EDITOR to 'emacsclient -a "" -nw' (and you might also want to alias emacs to that as well). What this does is connect to the emacs daemon. The -a "" starts one if it isn’t already started (you can also do this yourself with emacs --daemon. If you only want to use the daemon version if you’ve specifically started it, replace "" with emacs. This will connect to the daemon if it’s running, and otherwise just start a new emacs process.

The -nw keeps it from running in window mode. Remove this if you use the GUI version of emacs. This is necessary to make it work correctly with multiple tabs. This is so fast that you should never really even need to use C-z to quickly exit emacs. C-x C-c is just fine, because reopening will be instantaneous. I like this because I was starting to accumulate background emacs processes that I forgot about.

This probably requires a fairly new version of emacs, possibly even the development version.

Since I’m going cold-turkey, I am doing this over my break from classes, so that I can weed out any difficulties during a period when I can live with slow text editing if necessary. This is a one month break. I have reached (roughly) the half way point. For the first half, I used nothing but vim to edit text. Now, I will use nothing but emacs.

Now that I’ve stopped using vim (for now anyway), my view of it isn’t much different from what I wrote in the first part. A lot of things there were addressed by commenters (or rather commenter). I still feel that it’s not an a method of text editing that fits my head. My entire life, I’ve used text editors where typing inserts text, and various control characters do things like move around faster.

Enter emacs. It does exactly this. Also a ton more.

I’ve only been using emacs for two days, but here are my impressions so far:

  1. Move the cursor to the second line in the phrase below.
  2. Type  dd  to delete the line.
  3. Now move to the fourth line.
  4. Type   2dd   to delete two lines.

--->  1)  Roses are red,
--->  2)  Mud is fun,
--->  3)  Violets are blue,
--->  4)  I have a car,
--->  5)  Clocks tell time,
--->  6)  Sugar is sweet
--->  7)  And so are you.

>> Kill a line, move around, kill another line.
   Then do C-y to get back the second killed line.
   Then do M-y and it will be replaced by the first killed line.
   Do more M-y's and see what you get.  Keep doing them until
   the second kill line comes back, and then a few more.
   If you like, you can try giving M-y positive and negative
   arguments.

Well, that’s it for now. I will hold off on the questions until after I go through all the customizations, as it seems that, unlike vim, emacs has many things already built-in (but we already knew that, didn’t we :). So I have just one question for readers: does anyone know of a really good emacs cheatsheet? The one I used for vim was really awesome, but I haven’t found anything equal for emacs. I find myself searching the tutorial whenever I forget something, which is not very efficient, so I would appreciate something better. Otherwise, I’ll just find something decent and print it out, as it would be better than nothing.

And if anyone cares, you can see what I’ve got for my .emacs file so far at https://github.com/asmeurer/dotfiles/blob/master/.emacs.

Here’s an excerpt:

The concert hall at the Syndey Opera House holds 2,700 people. This blog was viewed about 11,000 times in 2011. If it were a concert at Sydney Opera House, it would take about 4 sold-out performances for that many people to see it.

Click here to see the complete report.

But I’d seen some of the nice features that vim has, like automatically keeping all lines under 80 characters, on a friend’s computer, and I decided that I should try it.

Now, I had had a little prior experience with both vim and emacs, but all that I remembered was for vim that i inserts and ZZ quits (for when I accidentally open it) and for emacs, that M-X doctor starts the psychiatrist.

So I’ve decided to try them out, doing it cold turkey. To make sure that I choose the better one, I’ve decided to try both. So, starting about a week ago, I’ve been using nothing but vim for all my text editing. Starting in January, I will try using emacs, and after two weeks, I will see what I like better.

My opinions so far on vim:

If there are any vim experts out there reading this, I have some questions:

EDIT: I thought of some more questions:

Finally, if anyone else is thinking of starting vim, I have some useful things I’ve already found in my .vimrc. So you might take a look at that, and add the ones that you like to your .vimrc. Finally, if you are on Mac OS X, you should use iTerm2. Actually, you should use this regardless of what text editor you use. It’s a very good Terminal.app replacement that has virtually all the features (with a couple of exceptions) as Terminal.app, and a ton of extra ones. The one I want to mention here is mouse reporting support, so you can use your mouse to do things in vim. This is very useful, as sometimes, e.g., when selecting text, using the mouse is just more efficient. Also, if you get frustrated trying to remember the commands that will move around you faster than h, j, k, and l, you can just click on where you want to go.

:wq

In [1]: sqrt(x)
Out[1]: x**(1/2)

In [2]: solve(x**2 - 2, x)
Out[2]: [-2**(1/2), 2**(1/2)]

Now suppose you took the output of those expressions and pasted them into isympy:

In [3]: x**(1/2)
Out[3]: x**0.5

In [4]: [-2**(1/2), 2**(1/2)]
Out[4]: [-1.41421356237, 1.41421356237]

That’s with __future__.division. Here’s what would happen with old division:

In [2]: x**(1/2)
Out[2]: 1

In [3]: [-2**(1/2), 2**(1/2)]
Out[3]: [-1, 1]

This is because with old division, 1/2 evaluates to 0.

The problem is that Python evaluates 1/2 to 0.5 (or 0) before SymPy has a change to convert it to a Rational. There were several ways that people got around this. If you copy an expression with number division in it and want to paste it into a SymPy session, the easiest way to do this was to pass it as a string to sympify():

In [1]: sympify("x**(1/2)")
Out[1]: x**(1/2)

In [2]: sympify("[-2**(1/2), 2**(1/2)]")
Out[2]: [-2**(1/2), 2**(1/2)]

If that was too much typing for you, you could use the S() shortcut to sympify()

In [3]: S("x**(1/2)")
Out[3]: x**(1/2)

In [4]: S("[-2**(1/2), 2**(1/2)]")
Out[4]: [-2**(1/2), 2**(1/2)]

This solution is fine if you want to paste an expression into a SymPy session, but it’s not a very clean one if you want to paste code into a script. For that, you need to modify the code so that it no longer contains Python int/Python int. The easiest way to do this is to sympify one of the ints. So you would do something like

In [5]: x**(S(1)/2)
Out[5]: x**(1/2)

In [6]: [-2**(S(1)/2), 2**(S(1)/2)]
Out[6]: [-2**(1/2), 2**(1/2)]

This wasn’t terribly readable, though. The best way to fix the problem when you had a power of one half was to use sqrt(), which is a shortcut to Pow(…, Rational(1, 2)).

Well, this last item should make you think. If sqrt(x) is more readable than x**(S(1)/2) or even x**(1/2), why not print it like that in the first place. Well, I thought so, so I changed the string printer, and now this is the way that SymPy works. So 90% of the time, you can just paste the result of str() or print, and it will just work, because there won’t be any **(1/2), which was by far the most common problem of “Python evaluating the expression to something before we can.” In the git master, SymPy now behaves like

In [1]: sqrt(x)
Out[1]: sqrt(x)

In [2]: solve(x**2 - 2, x)
Out[2]: [-sqrt(2), sqrt(2)]

You can obviously just copy and paste these results, and you get the exact same thing back. Not only does this make expressions more copy-and-pastable, but the output is much nicer in terms of readability. Here are some before and afters that come from actual SymPy doctests that I had to change after fixing the printer:

Before:
>>> e = ((2+2*sqrt(2))*x+(2+sqrt(8))*y)/(2+sqrt(2))
>>> radsimp(e)
2**(1/2)*x + 2**(1/2)*y

After:
>>> radsimp(e)
sqrt(2)*x + sqrt(2)*y

Before:
>>> b = besselj(n, z)
>>> b.rewrite(jn)
2**(1/2)*z**(1/2)*jn(n - 1/2, z)/pi**(1/2)

After:
>>> b.rewrite(jn)
sqrt(2)*sqrt(z)*jn(n - 1/2, z)/sqrt(pi)

Before:
>>> x = sympify('-1/(-3/2+(1/2)*sqrt(5))*sqrt(3/2-1/2*sqrt(5))')
>>> x
(3/2 - 5**(1/2)/2)**(-1/2)

After
>>> x
1/sqrt(3/2 - sqrt(5)/2)

And not only is sqrt(x) easier to read than x**(1/2) but it’s fewer characters.

In the course of changing this, I went ahead and did some greps of the repository to get rid of all **(S(1)/2), **Rational(1, 2) and similar throughout the code base (not just in the output of doctests where the change had to be made), replacing them with just sqrt. Big thanks to Chris Smith for helping me catch all instances of this. Now the code should be a little easier to read and maintain.

Future Work

This is a big change, and I believe it will fix the copy-paste problem for 90% of expressions. But it does not solve it completely. It is still possible to get int/int in the string form of an expression. Only powers of 1/2 and -1/2 are converted to sqrt, so any other rational power will still print as a/b, like

In [1]: x**Rational(3, 2)
Out[1]: x**(3/2)

Also, as you may have noticed in the last example above, a rational number that sits by itself will still be printed as int/int, like

In [2]: (1 + x)/2
Out[2]: x/2 + 1/2

Therefore, I’m leaving the issue for this open to discuss potential future fixes to the string printer. One idea is to create a root function that is a shortcut to root(x, a) == x**(1/a). This would work for rational powers where the numerator is 1. For other rational powers, we could then denest these with an integer power. It’s important to do this in the right order, though, as they are not equivalent. You can see that SymPy auto-simplifies it when it is mathematically correct in all cases, and not when it is not:

In [3]: sqrt(x**3)
Out[3]: sqrt(x**3)

In [4]: sqrt(x)**3
Out[4]: x**(3/2)

Thus but (to see this, replace with -1).

So the idea would be to print Pow(expr, Rational(a, b)) as root(expr, b)**a.

The merits of this are debatable, but anyway I think we should have this root() function in any case (see issue 2643).

Another idea, which is probably not a good one, is to always print int/int as S(int)/int. So we would get

>>> Rational(1, 2)
S(1)/2
>>> x**Rational(4, 5)
x**(S(4)/5)

This is probably a bad idea because even though expressions would always be copy-pastable, they would be slightly less readable.

By the way, in case you didn’t catch it, all of these changes only affect the string printer. The pretty printer remained unaffected, and would under any additional changes, as it isn’t copy-pastable anyway, and already does a superb job of printing roots.

At the end of that post, I noted that Ondřej and I had hacked it to make the colors more livable. Well, a couple of weeks ago, GitHub user jtriley sent me an email asking me to back port my changes.

A lot has changed since I wrote my blog post last year. PuDB now has an official mailing list and an official GitHub repo.

So I deleted my GitHub clone and reforked from the official version.

A lot has also changed in the official code. Andreas had added config support, including a built-in prefs dialog that lets you set a few settings: the ability to turn on or off line numbers and the ability to change themes.

So I took the new code and added my theme as an official theme. This was pretty straight forward to do.

But then, I got a little carried away.

I noticed that it was difficult to choose a theme with the built-in prefs window because you had to close and reopen the window each time you made a change. So I added code to make it auto-update your changes as you made them.

Then I went back and looked at my original blog post and looked at the things that I didn’t like. There were two things. First, the default stringifier for variables is type, which is completely useless. This is because type is very fast and stable to compute. I had previously hacked this to be str, but now that there was an official config file with a prefs dialog, I figured it should go there.

So I added support to change this setting. But this wasn’t enough for me. I also added the ability to define your own custom stringifier. You just create a Python file that defines a function called pudb_stringifier(obj), which converts obj into the desired string representation. I included an example file that gives a fancy example that uses signals to compute the string value, but times out after one second and falls back to the type. This alleviates one of the problems of using str, which is that it can be slow for objects with large string expressions, especially SymPy objects, where sometimes the printer can be slow.

The second thing I didn’t like was that although you can change the width of the right-hand side bar, you could not change the relative heights of the variables, stack, and breakpoints boxes. I never use breakpoints, and rarely use the stack, so I would prefer to have those smaller and the variables larger. So I implemented it so that the [ and ] keys make the selected view smaller or larger. This information is all saved in the config file, so it’s remembered when you close and reopen PuDB.

There was one other thing that I didn’t like, which a change since my last blog post that reversed the order of the stack variables from what it was. It used to be most recent at the bottom, but it was changed to most recent at the top. This perhaps makes more sense, but the buttons to move around the stack, u and d, were still the same: u moves down the stack (i.e., less recent), and d moves up. These keys were already well established—indeed, these are the same keys used in Python’s built-in debugger pdb—so I added a setting to change the stack order. This was an easy change to make at this point, as I was already well aquatinted with the settings code, and only two lines of code needed to be changed when the setting changed. Like all other settings, this uses the cool magic that changes the setting in real time, so you can see the effect without closing the settings window.

Then someone on the mailing list requested a feature that I realized I also wanted, the ability to wrap variables. Previously, any variable that was longer than the variable view would just be cut off. You could make it wider, but that only helped a little bit. Otherwise, if you wanted to see the whole variable, you had to open IPython by pressing ! and view it there.

So, I implemented this. This was definitely the hardest thing to implement. I found out that it’s ironically very difficult to debug PuDB itself. You can’t run PuDB inside of PuDB if PuDB crashes, as both instances will just crash. Also, PuDB eats any print statements. The solution, suggested by PuDB author Andreas Klöckner, was to get the ttys file of another terminal (e.g., /dev/ttys012) and write the output to that.

I also made it so that non-wrapped variables show ... at the end, at Andreas’s suggestion. I wanted to use the unicode …, but this was not working at all. I discovered how much unicode really is a mess in Python 2. The problem has something to do with … being a three byte character, and I think it also has to do with the color codes that urwid uses. I’ll try it again once PuDB is ported to Python 3, but for now, we are going to have to do with the three ascii dots.

The wrapping code is waiting for merge, but the rest are already in. Here is a screen shot demonstrating some of the things I did:

Click for full size image

Things that I implemented to notice here:

- The midnight theme.
- The stack and breakpoints views have been shrunken.
- The variables are wrapped.
- Wrapping for the variable fourhundred has been turned off (you can turn wrapping on or off on a per-variable basis by selecting the variable and pressing w). Notice that there is an ellipsis at the end to note it has been cut off.
- Nested variables now have | before them, to distinguish them from wrapped variables, which are also indented. This change may or may not be accepted by Andreas.

Here’s a screen shot showing the prefs window. I did not implement this, but I did implement all but the first two preferences in the window. I’ve made my window tall so you can see all the options. You really have to get the code and try it to see the auto-update awesomeness. You can open the prefs window by pressing Ctrl-p (this was not at all obvious to me the first time I used it, so I also submitted a patch that makes it open the first time you use PuDB).

Click to see full size image

So if you’re not already using this awesome Python debugger, you should. You can pip install pudb, or fork it at GitHub.

Running it in your code is very easy. Just add

import pudb;pudb.set_trace()

in your code wherever you want to set a break point, or you can do python -m pudb.run script.py.

This awesome tool has increased my productivity tenfold since I discovered it, and has helped me track down bugs that would have otherwise extremely difficult if not impossible to find. And now, it’s just better.

PuDB uses the urwid library to do all its console GUI magic. This library makes it pretty easy to do a lot of stuff. For example, it automatically does relative sizing of widgets, so, for example, when you resize the variables, stack, or breakpoints views, you are actually increasing the relative size of each, not the size in characters. This makes it portable against any terminal size. The library also made coding the prefs window autoupdate magic very easy.

Also, I just want to note that git and GitHub make collaboration like this very easy. I just forked his project, made some improvements, and submitted them as pull requests. Then it was easy to discuss the changes. If the code had not been on GitHub and especially if it had not been in git, I probably would have never bothered to submit my contributions upstream. I highly recommend that every open source project use git and GitHub.

SymPy 0.7.1 has been released on July 29, 2011. It is available at

http://sympy.org

The source distribution can be downloaded from:
http://code.google.com/p/sympy/downloads/detail?name=sympy-0.7.1.tar.gz

You can get the Windows installer here:
http://code.google.com/p/sympy/downloads/detail?name=sympy-0.7.1.win32.exe

And the html documentation here:
http://code.google.com/p/sympy/downloads/detail?name=sympy-0.7.1-docs-html.zip

About SymPy

SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible. SymPy is written entirely in Python.

Release notes

Major changes

- Python 2.4 is no longer supported. SymPy will not work at all in
Python 2.4. If you still need to use SymPy under Python 2.4 for some
reason, you will need to use SymPy 0.7.0 or earlier.

- The Pyglet plotting library is now an (optional) external dependency.
Previously, we shipped a version of Pyglet with SymPy, but this was
old and buggy. The plan is to eventually make the plotting in SymPy
much more modular, so that it supports many backends, but this has not
been done yet. For now, still only Pyglet is directly supported.
Note that Pyglet is only an optional dependency and is only needed for
plotting. The rest of SymPy can still be used without any dependencies
(except for Python).

- isympy now works with the new IPython 0.11.

- mpmath has been updated to 0.17. See the corresponding mpmath release
notes at http://mpmath.googlecode.com/svn/trunk/CHANGES.

- Added a Subs object for representing unevaluated substitutions. This
finally lets us represent derivatives evaluated at a point, i.e.,
`diff(f(x), x).subs(x, 0)` returns `Subs(Derivative(f(_x), _x), (_x,), (0,))`.
This also means that SymPy can now correctly compute the chain rule
when this functionality is required, such as with `f(g(x)).diff(x)`.

Hypergeometric functions/Meijer G-Functions

- Added classes hyper() and meijerg() to represent Hypergeometric and Meijer G-functions, respectively. They support numerical evaluation (using mpmath) and symbolic differentiation (not with respect to the parameters).

- Added an algorithm for rewriting hypergeometric and meijer g-functions in terms of more familiar, named special functions. It is accessible via the function hyperexpand(), or also via expand_func(). This algorithm recognises many elementary functions, and also complete and incomplete gamma functions, bessel functions, and error functions. It can easily be extended to handle more classes of special functions.

Sets

- Added FiniteSet class to mimic python set behavior while also interacting with existing Intervals and Unions

- FiniteSets and Intervals interact so that, for example `Interval(0, 10) – FiniteSet(0, 5)` produces `(0, 5) U (5, 10]`

- FiniteSets also handle non-numerical objects so the following is possible `{1, 2, ‘one’, ‘two’, {a, b}}`

- Added ProductSet to handle Cartesian products of sets

- Create using the `*` operator, i.e. `twodice = FiniteSet(1, 2, 3, 4, 5, 6) * FiniteSet(1, 2, 3, 4, 5, 6) or square = Interval(0, 1) * Interval(0, 1)`

- pow operator also works as expected: `R3 = Interval(-oo, oo)**3 ; (3, -5, 0) in R3 == True`

- Subtraction, union, measurement all work taking complex intersections into account.

- Added as_relational method to sets, producing boolean statements using And, Or, Eq, Lt, Gt, etc…

- Changed reduce_poly_inequalities to return unions of sets rather than lists of sets

Iterables

- Added generating routines for integer partitions and binary partitions. The routine for integer partitions takes 3 arguments, the number itself, the maximum possible element allowed in the partitions generated and the maximum possible number of elements that will be in the partition. Binary partitions are characterized by containing only powers of two.

- Added generating routine for multi-set partitions. Given a multiset, the algorithm implemented will generate all possible partitions of that multi-set.

- Added generating routines for bell permutations, derangements, and involutions. A bell permutation is one in which the cycles that compose it consist of integers in a decreasing order. A derangement is a permutation such that the ith element is not at the ith position. An involution is a permutation that when multiplied by itself gives the identity permutation.

- Added generating routine for unrestricted necklaces. An unrestricted necklace is an a-ary string of n characters, each of a possible types. These have been characterized by the parameters n and k in the routine.

- Added generating routine for oriented forests. This is an implementation of algorithm S in TAOCP Vol 4A.

xyz Spin bases

- The represent, rewrite and InnerProduct logic has been improved to work between any two spin bases. This was done by utilizing the Wigner-D matrix, implemented in the WignerD class, in defining the changes between the various bases. Representing a state, i.e. `represent(JzKet(1,0), basis=Jx)`, can be used to give the vector representation of any get in any of the x/y/z bases for numerical values of j and m in the spin eigenstate. Similarly, rewriting states into different bases, i.e. `JzKet(1,0).rewrite(‘Jx’)`, will write the states as a linear combination of elements of the given basis. Because this relies on the represent function, this only works for numerical j and m values. The inner product of two eigenstates in different bases can be evaluated, i.e. `InnerProduct(JzKet(1,0),JxKet(1,1))`. When two different bases are used, one state is rewritten into the other basis, so this requires numerical values of j and m, but innerproducts of states in the same basis can still be done symbolically.

- The `Rotation.D` and `Rotation.d` methods, representing the Wigner-D function and the Wigner small-d function, return an instance of the WignerD class, which can be evaluated with the `doit()` method to give the corresponding matrix element of the Wigner-D matrix.

Other changes

- We now use MathJax in our docs. MathJax renders LaTeX math entierly in
the browser using Javascript. This means that the math is much more
readable than the previous png math, which uses images. MathJax is
only supported on modern browsers, so LaTeX math in the docs may not
work on older browsers.

- nroots() now lets you set the precision of computations

- Added support for gmpy and mpmath’s types to sympify()

- Fix some bugs with lambdify()

- Fix a bug with as_independent and non-commutative symbols.

- Fix a bug with collect (issue 2516)

- Many fixes relating to porting SymPy to Python 3. Thanks to our GSoC
student Vladimir Perić, this task is almost completed.

- Some people were retroactively added to the AUTHORS file.

- Added a solver for a special case of the Riccati equation in the ODE
module.

- Iterated derivatives are pretty printed in a concise way.

- Fix a bug with integrating functions with multiple DiracDeltas.

- Add support for Matrix.norm() that works for Matrices (not just vectors).

- Improvements to the Groebner bases algorithm.

- Plot.saveimage now supports a StringIO outfile

- Expr.as_ordered_terms now supports non lex orderings.

- diff now canonicalizes the order of differentiation symbols. This is
so it can simplify expressions like `f(x, y).diff(x, y) – f(x,
y).diff(y, x)`. If you want to create a Derivative object without
sorting the args, you should create it explicitly with `Derivative`,
so that you will get `Derivative(f(x, y), x, y) != Derivative(f(x, y), y, x)`.
Note that internally, derivatives that can be computed are always
computed in the order that they are given in.

- Added functions `is_sequence()` and `iterable()` for determining if
something is an ordered iterable or normal iterable, respectively.

- Enabled an option in Sphinx that adds a `source` link next to each function, which links to a copy of the source code for that function.

In addition to the more noticeable changes listed above, there have been numerous other smaller additions, improvements and bug fixes in the ~300 commits in this release. See the git log for a full list of all changes. The command `git log sympy-0.7.0..sympy-0.7.1` will show all commits made between this release and the last. You can also see the issues closed since the last release [here](http://code.google.com/p/sympy/issues/list?can=1&q=closed-after%3A2010%2F6%2F13+closed-before%3A2011%2F7%2F30&sort=-closed&colspec=ID+Type+Status+Priority+Milestone+Owner+Summary+Stars+Closed&cells=tiles).

Authors

The following people contributed at least one patch to this release (names are given in alphabetical order by last name). A total of 26 people contributed to this release. People with a * by their names contributed a patch for the first time for this release. Five people contributed for the first time for this release.

Thanks to everyone who contributed to this release!

* Tom Bachmann
* Ondřej Čertík
* Renato Coutinho
* Bill Flynn
* Bradley Froehle*
* Gilbert Gede
* Brian Granger
* Emma Hogan*
* Yuri Karadzhov
* Stefan Krastanov*
* Ronan Lamy
* Tomo Lazovich
* Sam Magura*
* Saptarshi Mandal
* Aaron Meurer
* Sherjil Ozair
* Mateusz Paprocki
* Vladimir Perić
* Mario Pernici
* Nicolas Pourcelot
* Min Ragan-Kelley*
* Matthew Rocklin
* Chris Smith
* Vinzent Steinberg
* Sean Vig
* Thomas Wiecki

• June 28, 2011: SymPy 0.7.0 final is released.
• June 29, 2011: I type git merge sympy-0.7.0 in my integration3 branch.
• July 24, 2011 (today; technically July 25 because it’s 2 AM): I finish merging sympy-0.7.0 into integration3.

That’s right, it took me over three weeks—almost a month—to merge sympy-0.7.0 into integration3 (granted, I worked on other things at the same time, such as the SciPy 2011 conference, but to me, any merge that takes longer than a day to complete is a problem). This is because git decided that I needed to fix as a merge conflict just about every single change in the release branch since the base of integration3. The total was over 100 files. You can see the final merge commit here.

So I started git mergetool, without which this whole ordeal would have been 10 times worse. The mergetool, which on my computer is opendiff, i.e., File Merge, gave the correct change by default in most cases, so I actually did not have to manually fix the majority of the conflicts. But I did have to go through and do a lot of them. I had to manually check each difference in the polys, as I had made several changes there in the course of working on integration3. In several occisaions, I had to research a change using git log -S and fancy methods. And I noticed at least two regressions in the polys, which I fixed.

mergetool was useless against risch.py and test_risch.py, because in my branch I had renamed these to heurisch.py and test_heurisch.py. Fortunately, these were not really modified much by me, so I could basically just replace them with the sympy-0.7.0 versions.

Once I finished merging I had to deal with test failures. This was partly expected, as my branch has always had test failures due to my hack disabling algebraic substitution in exp, which is required for risch_integrate() to work, but there were also several unrelated ones.

Some of these were caused by wrong merge conflict resolutions by me. So I went through git diff sympy-0.7.0 change by change and made sure that nothing was different that I didn’t want to be. I would recommend doing this for any big merge.

Then, I had to fix a few bugs that caused test failures. Several semantics were changed in the release. I think the ones that I had to change were the renaming of has_any_symbols to just has, the renaming of Poly.as_basic() to Poly.as_expr(), and the swapping of the meanings of quo and exquo in the polys. There were also some doctest failures due to the change to lexicographic ordering in the printer.

After all that, there were two regressions that caused test failures. The first was the following:

Before:

In [1]: Integral((exp(x*log(x))*log(x)), x).subs(exp(x*log(x)), x**x)
Out[1]: 
⌠             
⎮  x          
⎮ x ⋅log(x) dx
⌡             

After:

In [1]: Integral((exp(x*log(x))*log(x)), x).subs(exp(x*log(x)), x**x)
Out[1]: 
⌠                    
⎮  x⋅log(x)          
⎮ ℯ        ⋅log(x) dx
⌡                    

This substitution is necessary because the Risch algorithm requires expressions like to be rewritten as before it can integrate them, but I try to convert them back after integrating so that the user gets the same thing in the result that he entered. I created issue 2571 for this.

The second was that I had several places in my docstrings with things like

Given a derivation D on k[t] and f, g in k(t) with f weakly normalized with respect to t, either raise NonElementaryIntegralException, in which case the equation Dy + f*y == g has no solution in k(t), or the quadruplet (a, b, c, h) such that a, h in k[t], b, c in k, and for any solution y in k(t) of Dy + f*y == g, q = y*h in k satisfies a*Dq + b*q == c.

The problem here is the “raise NonElementaryIntegralException,” part. The code quality checker things that this is an old style exception (like raise Exception, message), due to a poorly formed regular expression. I fixed this in a pull request.

The good news is that now a lot of stuff works that didn’t before because of fixes that were required that only existed in master. For example, the following did not work before, but now does due to improvements to RootSum:

In [1]: risch_integrate(1/(exp(5*x) + exp(x) + 1), x)
Out[1]: 
           ⎛    2                                                           
x + RootSum⎝21⋅z  + 6⋅z + 1, Lambda(_i, _i*log(-3381*_i**4/4 - 3381*_i**3/4 

                                       ⎞          ⎛     3        2          
- 625*_i**2/2 - 125*_i/2 + exp(x) - 5))⎠ + RootSum⎝161⋅z  + 115⋅z  + 19⋅z + 

                                                                            
1, Lambda(_i, _i*log(-3381*_i**4/4 - 3381*_i**3/4 - 625*_i**2/2 - 125*_i/2 +

             ⎞
 exp(x) - 5))⎠

In [2]: cancel(risch_integrate(1/(exp(5*x) + exp(x) + 1), x).diff(x))
Out[2]: 
      1      
─────────────
 5⋅x    x    
ℯ    + ℯ  + 1

The general definition of the logarithmic part of an integral is a sum over the roots of a polynomial, which must be expressed as a RootSum in the general case. Previously, RootSum.diff did not work, but thanks to Mateusz, an algorithm for computing exactly the RootSum where the Lambda expression is a rational function was implemented (see this bit from our SciPy tutorial for an idea on how this works), so now the Risch Algorithm can work with RootSum objects just as well with as an ordinary sum of logarithms.

Also, there was a bug in the square free algorithm in my branch that was fixed in master that was causing wrong results (I don’t remember the expression that produced them right now), and also there was a fix by me in master to make is_rational_function() faster, as it was significantly slowing down the calculation of some integrals (for example, risch_integrate(Add(*(exp(i*x) for i in range(1000)))), which is still slow to calculate, but now it’s because of other things).

About big branches

So this merge, along with the poly12 fiasco (which by the way, I think part of the reason git made me do all these merge conflict resolutions was because polys12 was rebased from the polys11 I had merged into integration3), has shown me very clearly that it is very bad to go off with your own branch and do a lot of work and wait a long time before merging it back into the main repo.

This is what was done with polys12. Mateusz had a lot of new polynomials code that he developed in one big branch, and when it finally came to merging it back in, it was a mess. This was for several reasons, which I do not want to discuss too much here, but it became clear to everyone I think that doing this was bad, and that it would have been better to have submitted many changes as pull requests as they were made than keeping them all together in one big branch for a long time.

This model also affected my work, as I had to work off of latest the polys branch, not master, as my work relied heavily on the latest and greatest in the polys.

Well, with this merge of the main repo into my branch, I see that my branch is starting to become the same way. I originally thought that I should finish the Risch algorithm before submitting it to be merged into master. I know know that this is the wrong approach. Development in master is too fast to keep code away from it for too long. The divergence makes it more and more difficult to merge back with every time. Furthermore, there are regressions that were never noticed to be regressions because the code that would have shown them existed only in my branch. Now I have to fix these, whereas if the code were in master, the regression would have never happened in the first place, because the author would have seen it immediately from the test failures.

I also thought that I should wait to merge because there were so many bugs in my code. But I see now that this is also wrong. Merging with master will help me find these bugs, as people will actually use my code. Sure, I’ve asked people to try out risch_integrate(), and some people have (and I thank you), but having it in the default integrate() in master will result in finding more bugs in the code than I ever would alone, which is basically the way it is right now with the code living only in my own branch.

I would prepare my code for merging with master today, if it weren’t for this exp.subs hack, which causes test failures and is technically a regression, but is required for the preparsing code to the Risch algorithm to work. This is why I wrote to the list two weeks ago asking for advice on how to structure the substitution code so that we can nicely have various kinds of substitutions (e.g., exact like I need and algebraic like currently exists in exp) living together without cluttering up the code.

Therefore, I am going to focus my energies on fixing this subs problem so I can get my code merged with master. Then, when this is done, I will continue my work on implementing the remaining cases of the Risch algorithm.

So let this tale be a warning to people working on a lot of code in a big branch. This especially applies to our GSoC students, as it’s extremely easy to let your code accumulate when you’re a GSoC student (technically this branch of mine is a GSoC branch). I see that some of our students are doing a better job of this than others. To those who have your code all in one big branch that hasn’t been merged, I recommend you ready your branch for merge now. And in the future, try to break your code up into small but still meaningful chunks and submit those as pull requests. With git, it’s easy to base the code you are currently working on on code that hasn’t been merged yet, while still keeping things in small chunks for the pull requests.

On the other hand, git will only take you so far if you keep everything in a big branch, because there are going to be changes in master that will affect your work, no matter how isolated you think it is, and these are the sorts of things that it is impossible for git to fix for you. But if your code is in master, it will be supported by everyone, and any major change that affects it will have to fix it. For example, if someone changes a printer and the doctests change, then he will have to change your doctest too if it’s in master, but if it’s in your branch, then you will have to fix it when you next merge/rebase with/against master.

• I met a ton of cool people. This included meeting several people who I had previously known from mailing lists in person for the first time. I met the SymPy developers Mateusz Paprocki and Andy Terrel, and I also had already known or heard about people like Fernando Perez, Gael Varoquaux, and Robert Kern. There are a lot of people out there who are excited to be using Python for their research, which is a real refresher from my university, where everyone is using Matlab and Maple.
• Mateusz and I gave a tutorial on SymPy. This was one of the four introductory track tutorials. This was a great experience to teach SymPy to people. You can see the Sphinx document that we used, and there should eventually be a video posted at the SciPy 2011 website.
• In addition to our tutorial, I attended some of the other tutorials. I particularly enjoyed the NumPy tutorial. Having never used NumPy before, I now feel comfortable with the basics. I also attended Gael Varoquaux’s tutorial on scikits.learn and Corran Webster’s tutorial on Matplotlib, Traits, and Chaco. My only regret is that the advanced track and introductory track tutorials were held at the same time, so I could not attend half of them. I plan to watch the ones I missed online.
• The general conference was excellent. Some of the talks that I particularly enjoyed were:
  • The keynotes. I found Eric Jone’s keynote particularly relevant as the leader of SymPy, as he talked about some of the good things to do and bad things to not do when leading a scientific project. I also enjoyed Perry Greenfield’s talk about how the astronomy community moved from some old proprietary system to Python.
  • Mateusz gave a talk on his FEMhub online lab, which a was very impressive system for using Python entirely in the web browser.
  • By far the best talk of the entire conference was Fernando Perez’s talk on the new IPython 0.11, which will be coming out in about a week or so. His demo of the new features such as the QT console and html notebook were very impressive. If you want to watch just one video from the conference, I would recommend that one.
  • Mark Dewing gave a talk about a system he wrote using SymPy to do automated derivation of equations. The system is impressive, and contains some features that would be nice to backport to SymPy. He told me that he wants to do this, so follow the mailing list. You can see what he has so far on his derivation_modeling branch at GitHub.
  • The lightning talks. These are very short talks at the end of the conference that are only five minutes long. In addition to many interesting talks, both Mateusz and I gave a lightning talk. Mateusz gave a talk on SymPy Live, which he recently improved to do things like give LaTeX output, and I gave a talk on my work with the Risch algorithm. I would also highly recommend watch this talk once they post the videos.
  • Again, regrettably, I could not attend half of the talks because they were held at the same time. Fortunately, they filmed all of them, so I hope to watch them all online when they are posted (and I recommend that you do too).
• The sprints were a great time for getting together and hacking together. I worked with Min Ragan-Kelley to make isympy work with the new IPython. Having fixed this, I now want to release 0.7.1 very soon, so I used some of the time during the sprints getting ready for that. We already have preliminary release notes, and my hope is to create a release candidate on Monday (tomorrow). I also finished up my MathJax branch and finished reviewing and pushed in Tom’s first GSoC pull request, which has a lot of really cool stuff relating to converting hypergeometric functions and Meijer G-functions into standard elementary functions. This will all be in the release.

  Also at the sprints, Mateusz worked on an extension for our Sphinx docs that puts a SymPy Live console right in the docs. You can then click on “evaluate” next to any of the code examples, and it will run it in SymPy live. And of course, you can then edit it and play around with it. He already had a working version of this by the end of the sprints (with a few bugs still), but I don’t think he has pushed it to GitHub yet. I think this is going to be a landmark change for our documentation. SymPy Live runs on the App Engine, so this approach can be applied to any library that can run in pure Python 2.5, and I think a lot of such projects are going to be jealous this and want to start using it, because it’s very impressive and useful.

  We also had a couple of people from the conference come to our table and work on SymPy. These were people who were new to SymPy, and I think attended our tutorial. One of them, Emma Hogan, worked a little bit on improving our documentation, and has submitted a pull request.

• Austin, TX is a nice city with lots of fun places to go, but it is also very humid, which is something I could barely stand (I am used to the same heat, but in Albuquerque it is dry heat). One interesting thing that some of us went and saw was the bats. The bridge over this lake in Austin has over a million bats living under it, and at night they all fly out to feed.

There’s all kinds of fun and interesting stuff that happened that I did not mention here. If you are interested in science and Python, I would highly recommend attending a future SciPy conference.

SymPy 0.7.0 has been released on June 28, 2011. It is available at

http://sympy.org

The source distribution can be downloaded from:
http://sympy.googlecode.com/files/sympy-0.6.7.tar.gz

You can get the Windows installer here:
http://sympy.googlecode.com/files/sympy-0.6.7.win32.exe

And the html documentation here:
http://sympy.googlecode.com/files/sympy-0.6.7-docs-html.zip