Category Archives: Programming

Just learning Python and this is puzzling me

Is there no constants in Python? Otherwise why can you do this??

import numpy as np
np.nan = 0
print(np.nan == np.nan, np.isnan(np.nan))  # True False

The operator can return different type depending on input and you have to check the return type. And I guess someone thought it is a great idea designing it?

df = pd.DataFrame([{'Name': 'Alice'},
                   {'Name': 'Jack'},
                   {'Name': 'Helen'}],
                   index=['group1','group1','group2'])
print(type(df.loc['group1'])) # pandas.core.frame.DataFrame
print(type(df.loc['group2'])) # pandas.core.series.Series

print(type(df.loc[df.index == 'group2'])) # workaround to always get DataFrame

Another way to treat missing values (the most illogical as to me)

print(np.nan == np.nan, np.nan != np.nan) # False True

Format date in Batch yyyyMMdd

The usual approach to build a timestamp for a log files in a batch is to use standard Windows utilities date and time. However the format of the output depends on locale and it is almost not possible to make the script which runs on any machine. A solution might be to create a small java tool.

Source file (datetime.java):

import java.text.SimpleDateFormat;
import java.util.Calendar;
public class datetime {
  public static void main(String[] args) {
    System.out.println(new SimpleDateFormat("yyyyMMdd_HHmmss")
          .format(Calendar.getInstance().getTime()));
  }
}

Compilation (you need JDK for this):

javac datatime.java

The code will be compiled into datatime.class.

Use in a batch file:

java datetime>logdate.txt
set /p STAMP=<logdate.txt
set LOG=C:\logs\%STAMP%.log
call Main.bat > %LOG%

Personal File Sharing

If you need easy accessible personal file store, there are a lot of them on the web. Another solution is to use google blobstore service – a geek’s choise 🙂

import urllib
from google.appengine.ext import blobstore
from google.appengine.ext import webapp
from google.appengine.ext.webapp import blobstore_handlers
from google.appengine.ext.webapp.util import run_wsgi_app

class MainHandler(webapp.RequestHandler):
    def get(self):
        upload_url = blobstore.create_upload_url('/upload')
        self.response.out.write("""<html>
        <body style="font-family:arial,sans-serif;">
        <form action="%s" method="POST" enctype="multipart/form-data">
        Upload File:<br/>
        <input type="file" name="file"/>
        <input type="submit" name="submit" value="Submit"/>
        </form>""" % upload_url)
        for b in blobstore.BlobInfo.all():
            self.response.out.write("""<li>
                <a href="/serve/%s">%s</a>
                (<a href="/delete/%s">Delete</a>)"""
                % (str(b.key()), str(b.filename), str(b.key())))
        self.response.out.write("</body></html>")

class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
    def post(self):
        self.redirect('/')

class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
    def get(self, resource):
        resource = str(urllib.unquote(resource))
        blob_info = blobstore.BlobInfo.get(resource)
        self.send_blob(blob_info, save_as=blob_info.filename)

class DeleteHandler(webapp.RequestHandler):
    def get(self, resource):
        resource = str(urllib.unquote(resource))
        blob_info = blobstore.BlobInfo.get(resource)
        blob_info.delete()
        self.redirect('/')

def main():
    application = webapp.WSGIApplication(
          [('/', MainHandler),
           ('/upload', UploadHandler),
           ('/delete/([^/]+)?', DeleteHandler),
           ('/serve/([^/]+)?', ServeHandler),
          ], debug=True)
    run_wsgi_app(application)

if __name__ == '__main__':
    main()

Configuring Eclipse on Windows to Use With Google App Engine

The post “Configuring Eclipse on Windows to Use With Google App Engine” on Google has not been updated since 2008. Here is the updated version for Eclipse 3.7 (Indigo).

Before you start, download and install the following components:

Python Google App Engine Calculator

Couple of months ago, accidentally I found myself on a conference for Python developers. I thought “what the hell am I doing here”. Ok, now I got into an interesting project and have to learn Python… 🙂

My first program – Calculator:)

import webapp2

class MainPage(webapp2.RequestHandler):
    def get(self):
        # build a list of operations
        f = {'+': lambda x, y: str(float(x) + float(y)),
             '-': lambda x, y: str(float(x) - float(y)),
             '*': lambda x, y: str(float(x) * float(y)),
             '/': lambda x, y: str(float(x) / float(y)),
             'C': lambda x, y: "",
            }
        # get page parameters
        x = self.request.get('x')
        y = self.request.get('y')
        operator = self.request.get('operator')
        # calculate 
        result = ""
        try:
            result = f[operator](x, y)
        except ValueError:
            result = "Error: Incorrect Number"
        except ZeroDivisionError:
            result = "Error: Division by zero"
        except KeyError:
            pass
        # build HTML response
        buttons = "".join(["<input type='submit' name='operator' value='"
                           + o + "'>" for o in sorted(f.keys())])
        self.response.out.write("""<html>
            <body>
            <form action='/' method='get' autocomplete='off'> 
            <input type='text' name='x' value='%s'/><br/>
            <input type='text' name='y'/><br/> 
            %s 
            </form>
            </body>
            </html>""" % (result, buttons))

app = webapp2.WSGIApplication([('/', MainPage)], debug=True)

Activity Tracker

Do you have to fill timesheet weekly? I have to.. This small tool can help you to remember what you were doing last week 🙂

You start the tool and minimize it to tray. It tracks activity on your computer and logs it to a file, so you can later see what you were doing. The information, the program tracks every minute, is:

  • was there any activity (mouse or keyboard input),
  • current foreground program,
  • the title of the current foreground program.

Click on the program icon in the tray, opens it. You can pick a date, and you will see the 24 hours activity chart in the bottom, and time statistics in the tree.

The color meaning on the activity chart are the following:

  • White – no information, the computer was turned off or the program was not running.
  • Gray – idle time.
  • Blue – there was some activity.
  • Red – highlights the time when the program selected in the tree was active.

Source Code

https://bukhantsov.org/tools/ActivityTracker.zip

The 64 bit executable is in

ActivityTracker.zip\ActivityTracker\ActivityTracker\bin\x64\Release

Security

The data file is stored in “My Documents” and has name activitytracker.txt.

If other people has access to the file, please think twice before using the tool. The program will log everything you do and every minute. 🙂

TO-DO: Password protection. The data should be stored in password protected zip file. To start the program, the user will have to enter the password for the zip file. If there is no data file, user will have to enter “new” password which will be used for the zip file.

A bit of code

The code builds a histogram and then orders the array by frequency in descending order.

Histogram:

string[] list = { "x", "b", "b", "x", "x", "z", "c", "x", "b", "c" };
Dictionary<string, int> histogram = new Dictionary<string, int>();
foreach (string s in list)
{
  if (!histogram.ContainsKey(s))
    histogram.Add(s, 1);
  else
    histogram[s] += 1;
}

The following code demonstrates use of lambda expressions for sorting (it was new to me).

List<KeyValuePair<string, int>> sortedlist = new List<KeyValuePair<string, int>>(histogram);
sortedlist.Sort((firstPair, nextPair) => -firstPair.Value.CompareTo(nextPair.Value));

foreach (KeyValuePair<string, int> p in sortedlist)
{
  Console.WriteLine(p.Key + ": " + p.Value);
}

This can also be done using LINQ:

var sortedlist = from s in histogram orderby s.Value descending select s;

foreach (var p in sortedlist)
{
  Console.WriteLine(p.Key + ": " + p.Value);
}

Update, 2011-11-19

Some bugs were corrected
Some enhancements
The new source code uploaded

Alternatives 🙂

Of course, the program was written mostly for fun. Consider ManicTime if you really need activity tracking software.

Log file with timestamp in Windows

How to build the log file name with timestamp in the format Log_yyyy_MM_dd_HHmm.log.

Batch file

@echo off

REM get date in format yyyy-MM-dd
FOR /f "tokens=1-3 delims=- " %%a IN ("%DATE%") DO (SET filedate=%%a_%%b_%%c)

REM get time in format HHmm
FOR /f "tokens=1-2 delims=:" %%a IN ("%TIME%") DO (SET filetime=%%a%%b)

REM if time in format Hmm - add leading zero
IF "%filetime:~0,1%"==" " SET filetime=0%filetime:~1,3%

REM prepare log file name in format Log_yyyy_MM_dd_HHmm.log
SET logfilename=C:\Logging\Log_%filedate%_%filetime%.log

REM do something redirecting result to %logfilename%

ECHO See the logfile: %logfilename%

Adjusting date pattern to correspond server settings

The above script assumes the server yyyy-MM-dd as the date format and Hmm as the time format (H stands for 24 hours time format). It uses command line tool DATE to get the current date and system environment variable %TIME% to get the current time. So the easiest way to make it work is to adjust server date and time short format to yyyy-MM-dd and Hmm in Region and Language settings. But this might be not the best solution, the other option is to adjust the script.

Let’s consider the line:

FOR /F "tokens=1-3 delims=- " %%a IN ('date /t') DO (SET filedate=%%a_%%b_%%c)

This line gets the date, parses it assuming format yyyy-MM-dd and builds the variable filedate in format yyyy_MM_dd.

FOR /F ["options"] %%variable IN (file-set) DO command [command-parameters]

This command reads and processes line by line all files from the files-set. You can read description of the command running FOR /?. In our case we get one line result of the command date /t and parse it using the specified options tokens and delims.

When option tokens is specified, additional variables are allocated. In our case we use first to third tokens, first token is assigned to variable a, second token is assigned to variable b, and third is assigned to variable c. We use these variables to set the variable filedate.

The option delims specifies the set of delimiters. In our case: space and minus signs.

So if your date format is ddd MM/dd/yyyy (e.g. Sat 09/10/2011), the command should be changed to

FOR /F "tokens=2-3 delims=/ " %%a IN ('date /t') DO (SET filedate=%%c_%%a_%%b)

The following line adds leading zero if necessary.

IF "%filetime:~0,1%"==" " SET filetime=0%filetime:~1,3%

Some other notes about batch files

DO part can be split over lines:

for /f "tokens=1-3 delims=- " %%a in ('date /t') do (
set year=%%a
set month=%%b
set day=%%c
)
set filedate =%year%_%month%_%day%

Or it can be written on one line with ampersand separator:

... do (set year=%%a&set month=%%b&set day=%%c)

To get last 3 characters you can use %var:-3%. To replace substring str1 with str2, you can use %var:str1=str2%.

Note that trailing space can be a part of variable. For example,

@echo off
set var =some text
echo "%var%" "%var %"
pause

This batch will print. %var% is not defined and thus empty.

"" "some text"

Further information

You can read more information about the batch commands in the help: FOR /? and SET /?.

Drawing tutor

There are many brain fitness exercises on the web that helps to develop mental skills. There are not that much programs that help to develop visual skills. Well I do not know any of such.

The purpose of this program is to develop visual skills. You could also improve your intellectual skills trying to figure out how to use it =).

There are four games:

  • Lines
  • Curves
  • Proportions
  • 3D

The basic principle behind most of them – you see a figure and a point on the left side, your task is to indicate location of the point on the right side. The complexity of the game is increasing with each correct answer and decreasing otherwise.

Click here to play Drawing tutor

The functionality is quite finished, but there is no fancy graphics etc. If you have ideas how to improve – please let me know.

Lines

Drawing tutor - Lines

Curves

Drawing tutor - Curves

Proportions

Drawing tutor - Proportions

3D

Drawing tutor - 3D