root/tags/REL3_5/CHANGES

Revision 13232, 40.9 kB (checked in by kasimir, 2 years ago)

Data only dump from old repository - all commits by kasimir

  • Property svn:executable set to *
Line 
1Copyright (c) 2006-2007, Joseph B. Kowalski
2Portions Copyright (c) 2007 Kasimir Gabert
3See LICENSE for licensing information
4
5TorStatus - Tor Network Status Changes
6
7===============================================================================
8Version 3 Changes {{{
9===============================================================================
10--Version 3.5--
11* Complete interface redesign
12* Integrated operating system detection in by default
13* Bandwidth change to have a progress bar
14* Legend created
15* WHOIS query integrated for IP addresses
16* Background colors for different router status
17* Overall softer colors
18* Added notifications for Asc or Desc sort orders
19
20--Version 3.4.2--
21
22* (ticket 12) Fixed error with server IP detection for non-Squid servers
23* (ticket 12) Added config option for Squid server
24
25--Version 3.4.1--                             (changes made by Kasimir Gabert)
26
27* Fixed bottom headers for new domain
28* Updated IP detection so it works through Squid
29* Added known mirrors bar
30
31TODO:
32* Move the two changes into configuration items
33* Centralize the known mirrors and make it dynamic
34
35--Version 3.4--                            (end of changes by Joseph Kowalski)
36Rename of the DNSBL server to DNSEL (DNS Exit List) server. This was by
37recommendation of Roger Dingledine of the Tor project. All filenames, variable
38names, etc, have been adjusted to reflect the rename.
39
40Added basic hit counting functionality to the DNSEL server. Now, every five
41minutes, the DNSEL server will write a record to the "DNSEL_LOG" table with a
42timestamp, a TotalResponses counter, and counters of how many "NOERROR",
43"SERVFAIL", "NXDOMAIN", and "NOTIMP" responses it has sent out since it was
44started. This procedure runs in it's own thread, and so should not have any
45measurable effect on performance.
46
47Improved the code that detects whether a user is accessing the site via a Tor
48hidden service. Previously, I checked to see if the user was coming from
49127.0.0.1. Now, I check to see if the "HOST" header provided by the client
50upon access exists in the "$Hidden_Service_URL" from the config file. If so,
51we tell them they are accessing the site as a hidden service. If not, we
52proceed on to the normal checks to see if they are accessing through the Tor
53network.
54
55Modified the DNSEL server to return SERVFAIL instead of NXDOMAIN for requests
56which are outside of the zone the DNSEL server is authoritative for.
57
58Modified the DNSEL server to return NXDOMAIN instead of NOTIMP for requests
59other than type 'A' and 'ANY' when the request is in the authoritative zone.
60
61Modified the documentation / default variables for the DNSEL server to use the
62"ip-port.torhosts.example.com" domain, rather than "tor.dnsel.example.com".
63This encourages users to setup servers that are consistent with the
64"torel-design.txt" document maintained by the Tor project.
65
66--Version 3.3--
67Complete rewrite of the DNSBL server in Java. This provided a performance
68increase on the order of approximately 5x over the old Perl version of the
69server. On a Pentium 2.4 GHz machine with 2 GBs of RAM, I was able to
70consistently obtain load test results of 200 queries/sec or more. These tests
71were conducted across the internet, and not on the local machine.
72
73Addition of the "dnsbl_server.php" page. This page provides usage instructions
74for the DNSBL server, and informs users of it's domain name. Note that the
75link to "dnsbl_server.php" from the main page will only display if you
76set the "$DNSBL_Domain" variable in the "config.php" file, so you can leave
77this variable set to 'null' if you are not running a DNSBL server, or do not
78wish to advertise it on the web site.
79
80Addition of the "$Hidden_Service_URL" variable to the "config.php" file. If
81you are also making the TorNetworkStatus site available through a Tor Hidden
82Service, you can enter the URL here, and a link will be added to the main page.
83If you are not making the site available as a Hidden Service, or do not wish
84to advertise it on the web site, you can leave this variable set to 'null'.
85
86Removal of the "tor_exit_status" page. This functionality is already provided
87by the main page (Tells a user if they are accessing the site through the Tor
88network or not), and the user already has two options to do custom exit
89queries if desired: The "tor_exit_query" page, and the DNSBL server. Both are
90linked to from the main page.
91
92Modifications to the "tns_update.php" file and the database create script to
93accommodate the re-write of the DNSBL server.
94
95--Version 3.2--
96Removed the first type of query that the DNSBL server could perform, which
97was to simply lookup if a provided IP address was an active Tor server or not.
98This was causing confusion amongst some users, and causing people to focus on
99whether or not an IP is a Tor server at all, which is exactly what we are
100trying to get away from by providing an answer to the more granular question
101"Is this IP address a Tor server that is capable of exiting to my destination
102IP and port?", which is precisely what the second (and now only) query type
103does. This had the added benefit or making the server a little faster, since
104a bit of logic was removed. Please see the "README_DNSBL" file in the source
105distribution for full setup and usage information.
106
107--Version 3.1--
108Nearly a complete redesign of the DNSBL server added in the last version to
109improve performance. Major changes:
110
1111) Previously, there was a Perl script which performed the actual listening
112for DNS requests, and when one was received, the script passed on the details
113of the request to a PHP script, which is where the main processing happened.
114The PHP script would then return a value to the Perl script indicating success
115or failure of the request. This had the horrible performance penalty of
116needing to invoke PHP every time a new request was received. Now, the entire
117process is handled within the same Perl script -- All of the PHP code to
118process the DNSBL request has been ported over to Perl -- So there is no
119calling an external executable again and again. Additionally, Perl runs faster
120than PHP with all other factors being equal.
121
1222) With the old PHP script, since it had to be invoked anew from within the
123Perl script with each request, this also meant that a new database connection
124had to be setup with each request. This also caused a performance penalty.
125Now, since everything is handled from within a single script, a single
126database connection can be maintained the entire time the script is running.
127
1283) Previously, when the DNSBL script did it's database queries to pull the
129IP addresses and exit policies of Tor routers, it did it the same way that the
130main application does, which means joining two large tables together on the
131router fingerprint key. While this works fine for the main application and
132it's very large queries, it did not suit the performance needs of a DNS
133server. Now, I have added two new in-memory database tables specifically for
134the DNSBL functionality. These two tables are populated during the update
135cycle, and contain only the necessary information (prejoined) required by the
136DNSBL server. The fact that they are in memory and are a lot smaller than the
137other tables should help the DNSBL server to return queries much quicker.
138These two tables act just like the set of NetworkStatus and Descriptor tables
139in that only one is ever active at a given time. The update cycle will always
140be updating the non-active table in the background while requests can continue
141being served by the active table. When the update is done successfully, a flag
142in the Status table is toggled to tell the application to begin using the
143other table as the active one, and the process repeats -- No downtime for
144updates.
145
146Fixed a bug that would cause the DNSBL server to only return "NXDOMAIN"
147responses if the database connection was lost until the script was restarted,
148even if the database connection came back up later. Now, the server will
149return "SERVFAIL" responses while the database server is down, but it will
150resume correct operation once the database server comes back up.
151
152Various other minor bug fixes, mainly relating to the "Tor Exit Status" and
153"Tor Exit Query" functionality that was added in the last release.
154
155--Version 3.0--
156Complete redesign of the way that the application determines if you are
157accessing the site through the Tor network or not, or if the provided IP
158belongs to a Tor server capable of exiting to the specified IP and port (See
159section on "Tor Exit Query" page, below). Previously, we would simply check to
160see if the IP address that the user was coming from was a Tor server with the
161'Exit' flag set to true, and tell them they were coming through the Tor
162network if so. Now, we check if the user is coming from the IP address of a
163Tor server, regardless of it's 'Exit' flag status. If so, we pull the Exit
164Policy for the matching Tor server(s), look at what IP and Port the client is
165accessing us on, and parse the exit policy to determine if the Tor server(s)
166would allow or deny such an access. We then give the user information on each
167Tor server that matches their exit IP, letting them know if it would allow
168access to us or not considering it's exit policy. Finally, we also tell the
169user if it appears that they are coming to the site via a Tor Hidden Service
170or from the local machine directly. This is determined by checking if the user
171is coming to the site from the '127.0.0.1' IP address. Note that the
172application now handles situations where there are multiple Tor servers with
173IP's that match where the user is coming from (Or the IP provided, in the case
174of the "Tor Exit Query" page, see below). In these cases, it will cycle through
175each applicable Tor server to provide Exit Policy match information.
176
177Addition of "Tor Exit Status" page. This page provides the same information
178about whether a user is accessing the site through the Tor network as is
179available on the main page, but without all the other information. This may be
180easier for someone who just wants a simple answer for "Is my Tor installation
181working?", and should serve as a good entry point for such situations. This
182page also contains a link to the main Tor documentation page.
183
184Addition of "Tor Exit Query" page. This page allows a user to enter in an IP
185to check if it matches that of one or more Tor servers, and, if it does,
186checks to see if those Tor servers are capable of exiting to the destination
187IP address and port that the user provides.
188
189Addition of a DNSBL server. This allows a standard, non-web interface method
190for clients to query whether or not an IP address belongs to an active Tor
191server, and whether or not an IP address belongs to an active Tor server that
192would allow exiting to a specified destination IP address and port. Please
193see the "README_DNSBL" file in the source distribution for full setup and
194usage information.
195===============================================================================
196}}}
197===============================================================================
198
199===============================================================================
200Version 2 Changes {{{
201===============================================================================
202--Version 2.0--
203
204Small performance enhancement to background update process. When clearing old
205data from the descriptor and network status tables, we now use the "truncate
206table" command rather than the "delete from" command. This is faster because
207"truncate table" deletes all records from the table at once, while "delete
208from" deletes records one row at a time.
209
210Enhancement of "Require Flags" custom query option so that the user now has
211three options available for each flag: "Off", "Yes", and "No". "Off" means
212the status of the flag is not a factor in the search. "Yes" means a router
213must have flag set in order to show up in result set. "No" means that a router
214must not have flag set in order to show up in result set. Previously, a user
215only had the equivalent of the "Off" and "Yes" options. Also, when a flag is
216set to "Off", it's column header in the result list will have a blue
217background, "Yes" will have a green background, and "No" will have a red
218background. This further increases the level of flexibility with searches.
219
220Addition of "Country Code" field to "Tor Server Providing Network Status
221Opinion" section. 
222
223Addition of background color to drop-down lists and edit box in advanced
224search section.
225
226Minor changes to text in custom query section to facilitate clarity.
227
228
229--Version 2.1--
230
231Addition of "Aggregate Network Statistic Graphs / Details" page. This page
232contains four graphs of network-wide aggregate information: Number of Routers
233by Country Code, Number of Routers by Observed Bandwidth, Number of Routers
234by Time Running, and an aggregate summary of various criteria including router
235flags.
236
237Addition of "EventDNS" and "Hibernating" statistics to newly renamed
238"Aggregate Network Statistics Summary" section on main page.
239
240Improvement to the way that the "Aggregate Network Statistics Summary" section
241of the main page gets it's data. Previously, each piece of information used
242was collected by a separate SQL query. Now, all of this has been joined into
243one large query to cut out the overhead of multiple SQL calls.
244
245Change to CSS which affects the highlight color of links which are on a white
246background, for increased readability.
247
248
249--Version 2.2--
250
251Fixed a bug in the code that detects whether or not the user is connecting to
252the server through the Tor network. Previously, it would tell a user they were
253connecting through the Tor network if their IP address matched the IP address
254of any Tor server. Now, it will only tell the user they are connecting through
255the Tor network if their IP address matches the IP address of a Tor server AND
256that Tor server is an exit node.
257
258Addition of exit node only CSV file download, for those who don't want the
259IP's of all Tor servers.
260
261Addition of graph of exit routers by country code to Network Detail page. This
262is useful for letting us see where we are most likely to exit the Tor network.
263
264Addition of "BadDirectory" flag information everywhere we make use of flags.
265The "BadDirectory" flag was added in Tor version 0.1.2.5-alpha.
266
267Minor cleanups to documentation to enhance readability.
268
269Fixed minor bug in SQL database create script.
270
271
272--Version 2.3--
273
274Redesign of the "Custom / Advanced Query Options" box on the main page. I am
275now using drop-down lists for the sort request, sort order, and address mode
276parameters instead of radio buttons, which takes up a lot less screen real-
277estate and also makes adding new options for these parameters in the future
278more practical.
279
280Slight changes to the "Application Server Details" (Previously "Application
281Server Information") box on the main page to enhance readability.
282
283Addition of the "Number of Descriptors In Cache" field to the "Application
284Server Details" box.
285
286Movement of the "Application Server Details" box to the bottom of the page.
287This has the dual purpose of improving page readability as well as making the
288"Approximate Page Generation Time" figure more accurate, since it is now
289calculated closer to the bottom of the page. Previously, the PHP code which
290generates the "Custom / Advanced Query Options" box still ran after this time
291was calculated.
292
293Update of source URL at bottom of page to new domain.
294
295
296--Version 2.4--
297Modified "Aggregate Network Statistic Summary" box to also show "percent of
298total network" values for each statistic listed. This allows you to easily see
299what percent of Tor routers are exit nodes, what percent are mirroring the
300directory, what percent of the total network your current result set makes up,
301etc.
302
303Modified "Network Status Opinion Source" box to allow clicking on the nickname
304of the router to jump directly to its router detail page. This makes it easy
305for a user to get detailed information about the source of the network data
306that they are seeing.
307
308Modified code that detects whether a user is accessing the site through the
309Tor network or not. Now, if the user is accessing through the Tor network, the
310name of their exit node is displayed in addition to the IP address. Also, the
311user can click on the name of the exit node to jump directly to its router
312detail page. This makes it easy for a user to get detailed information about
313the exit node they are currently coming through.
314
315Modified page footer to also include a link to the change history and license
316of the application.
317
318Removed the "$MinRouterThreshold" variable from the config file, and all code
319that made use of the variable. This really became obsolete when version 1.4 of
320the application was released and we started using two sets of tables in the
321database for storing router information instead of one. Since that time, a
322user will always have their page generated from the active set of router
323tables in the database, while the next refresh operation is being performed on
324the inactive set. This makes this variable, which was used to see if a
325database refresh was in progress and display an error to the user back when we
326used a single set of tables, obsolete.
327
328Fixed a bug in the generation of the Comma Separated Value (CSV) list of exit
329nodes. A comma was incorrectly being added after the last entry in the file,
330which might have caused problems for parsers.
331
332Added "Number of Routers by Platform" graph to network detail page. This lets
333us easily see how popular different operating systems are for running Tor
334routers.
335
336Modified "Number of Routers by Time Running (Weeks)" graph on network detail
337page to be full page width. This makes this graph easier to read since it
338usually has a lot of items along the x axis.
339
340
341--Version 2.5--
342Fixed lots of minor HTML and CSS errors. All pages now validate as "CSS
343version 2.1" and "HTML 4.01 Transitional".
344
345Modified the handling of all SQL queries which return a result set so that the
346result set populates into an associative array rather than a numerically
347indexed array. This allows items in the result set to be referred to by column
348name rather than a numerical array index. This is a significant step towards
349being able to offer the user the ability to do things such as select which
350columns they want displayed on the page, etc, as it is much more flexible
351than using numerical indexes because your entire result set handling code is
352no longer affected by re-arranging the order of items in your query, or adding
353and removing items on the fly. It is also generally considered a database
354programming best practice, as a lot of confusion and potential for bugs is
355eliminated by being able to use easily recognizable column names as opposed to
356numerical indexes.
357
358Modified the "Address Display Mode" in the "Custom / Advanced Query Options"
359so that users now have a third choice -- "Hostname + IP Address". Previously,
360the user could only choose one at a time, and had no way to view both
361simultaneously. "IP Address Only" mode is still the default as the "Hostname +
362IP Address" mode can result in a much larger amount of data being returned,
363but now the option is there for those who want it.
364
365
366--Version 2.6--
367Added the ability for users to select which columns they want displayed in the
368main router table. This should be good for ensuring that the page does not
369grow larger and larger as new informational items about Tor routers become
370available, since we can just maintain reasonable default settings and leave
371the obscure stuff off unless a user intentionally turns it on. Also, the SQL
372query for obtaining the main router list has been made dynamic so that it will
373only request columns from the database if the user will actually be displaying
374them.
375
376Added "Fingerprint" as a column display option.
377
378Removed "Address Mode" from "Custom / Advanced Query Options" since it is made
379obsolete by the users ability to select which columns they want displayed,
380which offers more granularity.
381
382Efficiency improvement in the way that it is determined if a given flag
383column for a given router should display the red "x" or green check mark.
384Previously, there was an "If" statement that set the appropriate CSS class for
385the table data cell based on whether a flag variable for that router was '0'
386or '1'. Now, there is a CSS class that ends in '0' and '1', and the variable
387value is just dumped directly into the class statement in the appropriate
388place. This saves up to '$NumberOfDifferentFlags X $NumberOfRoutersInResultSet'
389'If' statements from being executed on each page load, normally many thousand.
390
391Fixed a bug in the generation of CSV files that was causing a comma to be
392added after the last column in a row. According to RFC 4180, this is incorrect.
393Also, added a newline character (\n) to the end of files.
394
395Added "Comma Separated Value (CSV) List of Current Result Set" option. This
396lets a user get a comma separated value (CSV) file of their current search
397result set, exactly as displayed on screen (Including sort preferences, column
398display preferences, and any other custom query parameters that may be
399applied). The first record (row) in the generated CSV file will contain the
400column names for reference. This should be very useful for users wishing to
401export the data from a custom query into an external data source.
402
403
404--Version 2.7--
405Moved the column display options to their own page, rather than having them on
406the main page in the "Custom / Advanced Query Options" box. You can get to the
407column display options by following the "Custom / Advanced Display Options"
408link from the top of the page and then clicking on the "Column Display
409Preferences" link. This should provide a much more comfortable interface for
410customizing column display options.
411
412Added the ability for users to re-configure the order that columns display in.
413Previously, users could only add and remove columns. This should allow much
414more flexibility to users wishing to customize the look of the page.
415
416Updated "Comma Separated Value (CSV) List of Current Result Set" functionality
417to support the newly added ability of users to re-configure column display
418order. This way, the CSV file of the current result set benefits from the same
419level of flexibility as the main page display.
420
421
422--Version 2.8--
423Added the ability for users to display four additional columns on the main
424page: "Platform", "Contact", "EventDNS", & "Hibernating". Users can sort by
425the new columns and, in the case of the "EventDNS" and "Hibernating" items,
426filter query results based on whether they are true or false, as they are
427treated as "Flags" by the application. All four of the newly added columns are
428set to "Off" by default.
429
430Updated the "Comma Separated Value (CSV) List of Current Result Set" script to
431support the addition of the new columns.
432
433Bug fix to the "Comma Separated Value (CSV) List of Current Result Set" script:
434Commas (,) and double-quotes (") are now replaced with a dash (-) and a single-
435quote ('), respectively, when they occur in a field to be written to the CSV
436file, as these characters would cause problems for automatic parsers.
437
438Modified the "Router Detail" page to display information on the "EventDNS" and
439"Hibernating" items alongside the other flags, since it has now been
440determined that the application will be treating these items like any other
441flag. Previously, these two items were displayed in the "General Information"
442section of the page, not the "Router Flags" section.
443
444Modified the order in which some items display in the "Aggregate Network
445Statistic Summary" section of the main page and the "Aggregate Summary --
446Number of Routers Matching Specified Criteria" graph on the "Network Detail"
447page to enhance consistency throughout the application.
448
449Minor source code cleanups to enhance consistency between scripts.
450
451
452--Version 2.9--
453Various application-wide code cleanups to fix all known situations that would
454generate PHP warnings.
455
456Fixed a rare division by zero bug that could occur during the generation of
457the bandwidth history graphs on the router detail page.
458
459Updates to the way "Uptime" information is presented. Previously, the "Router
460Detail" page would calculate the real-time uptime of the router (By looking
461at the difference between the current time and when the router descriptor was
462last published, and adding this value to the published "Uptime" value in the
463descriptor, compensating for time zone differences), but the other places that
464made use of the "Uptime" value did not, such as the main page, the "Network
465Detail" page, and the "Comma Separated Value (CSV) List of Current Result Set"
466functionality. These places would simply use the static "Uptime" value from
467the latest descriptor, and would not perform any real-time calculations on it.
468Now, all places that make use of the "Uptime" value perform the real-time
469calculations on it. This eliminates discrepancies between what the "Router
470Detail" page shows and what is shown everywhere else. Additionally, code to
471gracefully deal with situations where a certain descriptors publish time is
472obviously wrong, such as when it's in the future, was added to all places
473doing the "Uptime" calculations.
474
475Modified all places that display a "DirPort" value to say "None" rather than
476"0" for routers that do not mirror the directory. This is less confusing.
477
478Modified all places that display a "CountryCode" value to say "N/A" rather than
479nothing for routers where country code information could not be retrieved.
480Again, less confusing.
481
482Made the "Tor Network Status" text at the top and bottom of each page a link
483to get back to the main page. This should be more user-friendly than having to
484hit "back" to get to the main page when on the "Network Detail" and "Router
485Detail" pages. The links also exist on the main page itself, which will just
486refresh the page with the user's current settings.
487
488Added two new array variables to the config file to allow the default column
489display options to be set on a global scale. These options are what will be
490used when the user has not yet performed any customization on their display
491preferences. Both the active/inactive columns and their display order can be
492specified.
493
494Changed default column display preferences -- Added "Hostname" and removed
495"Bad Dir" and "Bad Exit".
496
497Removed support for the "EventDNS" descriptor flag entirely. The logic for this
498functionality was faulty, and since all new Tor versions have EventDNS enabled
499by default, and the Tor developers plan to remove the old DNS logic entirely
500at some time in the future (According to the "dir-spec" document), I decided
501to just go ahead and remove this now since it is becoming irrelevant.
502
503Removed "IRIX" as a listed operating system on the "Platform Graph", as it
504seems there have been no Tor servers running on this OS for some time now.
505
506Other minor fixes/enhancements.
507===============================================================================
508}}}
509===============================================================================
510
511===============================================================================
512Version 1 Changes {{{
513===============================================================================
514--Version 1.0--
515
516Initial release.
517
518
519--Version 1.1--
520
521Addition of router detail page.
522
523Addition of real-time-calculated router uptime to router detail page.
524
525Addition of bandwidth graphs to router detail page. This functionality makes
526use of the free 'JPGraph' PHP libraries.
527
528Addition of country code data to main and router detail pages. This
529functionality makes use of the free 'GeoIP' PHP API and country database.
530
531Separation of database refresh operation from main page render operation. The
532database refresh operation has been moved out of the web root into it's own
533script, 'tns_update.php'. This greatly increases page render speed in
534situations where the local-cache is expired.
535
536Addition of 'tns_agent.php' script to handle background scheduled database
537refresh operations.
538
539Movement of the SQL database creation script out of the web root.
540
541Addition of ability to click on a column header on main page to sort. Multiple
542clicks alternate sort direction.
543
544Addition of code to handle situations where a user attempts to load a page
545while a background database refresh is in progress. In these situations, the
546user will be notified of the database refresh being in progress, and the page
547will automatically reload in 10 seconds, retaining any sort or search criteria
548that was initially passed by the user.
549
550Various performance improvements during main page render, some major. Code
551cleanups. Implementation of CSS / HTML restructuring to greatly reduce the
552size of page.
553
554
555--Version 1.2--
556
557Application-wide security fixes.
558
559Multiple XSS vulnerabilities fixed.
560
561Possible SQL injection vulnerability fixed.
562
563Code cleanups, further performance enhancements.
564                                               
565
566--Version 1.3--
567                       
568Major changes to the way the application gets it's network status information
569and router descriptors.
570
571Previously, for network status information, the application would cycle
572through the local Tor server's "cached-status" documents of Authority servers
573you explicitly allowed in the application config file, using a different one
574for each local-cache refresh. This had two problems. First, it meant that you
575would get a different number of routers listed, and oftentimes different
576properties listed for a specific router depending on which Authority server's
577cached-status doc had been used. Second, and this is the bigger of the two
578problems, is that this is not the way the Tor network works. The Tor network
579leaves it up to each client/server to calculate what it believes to be the
580correct view of the network based on the multiple sources of network status
581information it has available. This is important for security reasons, and
582plays a significant role in making it harder for someone to play some bad
583tricks on network users. So, now network status information is obtained by
584asking the local Tor server's control port to list all routers it has an
585opinion about. This ensures the data being presented has been through the
586proper "voting" process done by the Tor software, and is what the local Tor
587server believes to be correct.
588
589Previously, for individual router descriptors, the application would do one
590of two things. One, the default, was to directly ask whichever Authority
591server was used for the last local-cache refresh. This was bad, as it
592basically meant that for the duration of the current cache life, every
593visitor going to the "router_detail" page was causing a request for a
594descriptor to be sent to that Authority server. Not what we want for the
595Authority servers, I'm sure. Two, a user could specify that they always wanted
596to use a certain Tor server for descriptor requests. This fixed the issue with
597hammering the Authority servers, but only if people used the option. Now,
598router descriptors are also obtained through the local Tor server's control
599port. Also, router descriptors are no longer written to files on the hard
600drive. This was unnecessary and slow.
601
602These changes allowed a lot of clean-up to be done in the config file. There
603are a lot less variables in there now.
604
605 
606--Version 1.4--
607
608Addition of an option to export the IP Addresses of all current Tor routers to
609a CSV (Comma Separated Value) file. This may be of some use for people looking
610to get a quick, current list of Tor router IP's, whether to formulate block
611lists, or to assist with other Tor network status apps / pages.
612
613Addition of "Bandwidth" and "Uptime" columns to main page. The bandwidth
614reading is obtained from the "Bandwidth Observed" parameter of the router
615descriptor and is displayed in KB/s. The uptime reading is obtained from the
616"Uptime" parameter of the router descriptor and is displayed in days. Both of
617these columns are sortable.
618
619Back end changes to make the addition of the "Bandwidth" and "Uptime" columns
620to the main page, described above, possible. These changes consisted of making
621the scheduled background update process request and parse the descriptors of
622every router that the local Tor server has an opinion about, as identified by
623it's network status request response, so that the bandwidth and uptime
624information (and everything else in the descriptor) could be inserted into the
625database at this stage to be available for use on the main page. Previously,
626descriptors were only requested and parsed on an as-needed basis when a user
627loaded the "router_detail.php" page, and the data parsed from them was not
628stored. Now, it's all inserted into the database during the main update
629process which opens up the possibilities for all kinds on new stuff. The
630downside to doing it this way is that the main update process CAN take
631considerably longer (especially when the local Tor server is very busy), which
632led to the second major back end change: Using two router (local-cache) tables
633instead of one. Basically, only one table is marked as active at any given
634time, and all page loads use the active table. When the update process runs,
635it runs on the non-active table, and marks it as the active table upon
636successful completion. This prevents users from seeing the "Database refresh in
637progress" error message, as they will instead just be seeing the old data
638until the new data is ready. I was quite serious about not causing a
639performance hit to page load times due to the new functionality, and this was
640the solution implemented to avoid that.
641         
642
643--Version 1.5--
644
645Major performance enhancements to both the background update process and the
646main page load process.
647
648First, for the update process, we are now obtaining the full network status
649opinion document and the full recent descriptor document from the control port
650of the Tor server. Previously, we were obtaining the full network status
651opinion document, and then, as we parsed out this file, we requested server
652descriptors of the Tor server's control port individually. This tended to
653sometimes cause somewhat of a load on the Tor server and cause the descriptors
654to be fed back very slowly, sometimes taking up to a half hour for a full
655refresh. Now, since we grab both full documents all at once, it only takes the
656Tor server a few seconds to provide them. Second, we are now writing the
657parsed network status information and the parsed descriptor information into
658two separate database tables (NetworkStatus & Descriptor). This makes for
659cleaner design and tables with less columns. And, in keeping with the spirit
660of the updates made in Version 1.4, there is two of each of these tables
661(NetworkStatus & Descriptor) so that one set can be updated while the other
662set can continue to be used by folks loading the page and running queries,
663etc., until the new set is ready.
664
665Performance improvements to the load time of the main page were obtained by
666switching to the MySQL MyISAM storage engine, instead of the InnoDB engine we
667had been using previously. MyISAM is significantly faster than InnoDB, and,
668since we currently don't make use of transactions, there is no reason not to
669use it. Also, indexes were added to the NetworkStatus and Descriptor tables
670on the "Fingerprint" column, since this is what is used for the join of these
671two tables during page load. This further increased the query speed.
672
673Some other small performance improvements were implemented, especially on the
674"router_detail.php" page.
675
676The update process now has more robust error handling.
677
678Addition of "Count" column to main page.
679
680Addition of green check marks and red x's to indicate whether a flag is "Yes"
681or "No". Previously, it would just print "Yes" or "No".
682
683Modifications of font selections and background images to hopefully achieve
684two things: Fit more in a smaller space, and make the page easier to read.
685
686
687--Version 1.6--     
688
689The application now performs reverse-DNS lookups during the update process and
690stores this information in the database. This enables a user to choose whether
691they would like addresses displayed as IP or Hostname. Of course, whichever
692display they choose, it is sortable.
693
694Added Hostname information to the router detail page as well.
695
696Added "Last Update Elapsed Time" field to main page. Good way to see if your
697system is experiencing problems during the background updates.
698
699Added "Current Descriptor Signature" field to the main page for the Network
700Status Source router information section.
701
702Minor CSS changes to improve layout and formatting in certain situations. 
703
704
705--Version 1.7--
706
707Changes to the way variable handling is done. I moved all state maintaining
708variables into PHP session, such as all of the "Flags" variables and the
709Address Display Mode, SortRequest, and SortOrder variables. Also, wherever
710possible because of addition to session, variables were removed from the URL's
711in the anchor tags of the generated page. The only ones that were left in
712URL's were variables that need to be manipulated by clicking on a link, namely
713the SortRequest and SortOrder variables. This makes the URL's in the anchor
714tags of the generated page much shorter and cleaner, since a whole lot less is
715being passed around via GET. This in turn reduces the generated page size,
716sometimes considerably depending on the custom query options in effect.
717Another nice benefit to doing things this way is that if a user has custom
718options set, completely browses away from the site, and later comes back
719within their same browser session, all of their options will be configured as
720they were when they left.
721
722Also, variables that needed to remain in URL's have shorter names to further
723reduce the amount of data being passed around via GET, and hence the length
724of the URL's in the generated page.
725
726Further cleanups to HTML formatting and CSS on both the main page and the
727router detail page.
728
729Addition of timer to bottom of main page to indicate how long page generation
730took on the server.
731
732 
733--Version 1.8-- 
734
735Changes / cleanup to CSS. All CSS information is now defined in one external
736style sheet which resides in the "web/css" directory, and the different parts
737of the code that need the style sheet simply reference it rather than defining
738everything at the top of every HTML page output to the browser.
739
740Addition of row highlighting for Mozilla Firefox / Opera. This makes it easier
741to visually track which row you are currently moused over. This feature does
742not currently work in Internet Explorer since that would require JavaScript,
743and I'm trying to keep this app JavaScript free for those who browse with it
744disabled.
745
746When sorting by IP Addresses, they are now sorted naturally, rather than as
747strings. This should make the IP Address search quite a bit more meaningful.
748This change also applies to the CSV (Comma Separated Value) file list of IP's.
749
750
751--Version 1.9-- 
752
753Added proper escaping of problem characters to all strings that get populated
754from data parsed from router descriptors and network status documents before
755database inserts are performed. This should prevent the update process from
756dying when these characters are encountered, which would sometimes happen.
757
758Now using the standard "gethostbyaddr" PHP function to do hostname lookups.
759Previously, I was doing some custom parsing on the output of the Unix "host"
760command, as I thought it would be faster, but, after comparing the two, I see
761no speed degradation with using "gethostbyaddr". The advantage of using
762"gethostbyaddr" over "host" is that we are not using a platform-specific
763command which limits where the application can run without modification.
764
765Addition of Advanced Search functionality, which lets you perform free-text
766searches on the following data: Fingerprint, Router Name, Country Code,
767Bandwidth, Uptime, Last Descriptor Published, IP Address, Hostname, Onion
768Router Port, Directory Server Port, Platform, & Contact. You can apply the
769following modifiers when searching the above data: Equals, Contains, Is Less
770Than, or Is Greater Than. The Advanced Search works in conjunction with the
771other Custom Query options (Sort By, Sort Order, Address Display Mode, and
772Require Flags), allowing a great deal of flexibility to customize the result
773set as desired.
774
775Addition of indexes where necessary in the database to better facilitate
776speedy free-text searches.
777
778Addition of "Hostname" field to "Tor Server Providing Network Status Opinion"
779section.
780
781Addition of feature to tell user if they are accessing the page through the
782Tor network or not, color coded.
783
784Additional escaping of potentially troublesome HTML characters before
785displaying on page for "Contact" field on main page and router detail page.
786===============================================================================
787}}}
788===============================================================================
789
790NOTE: This product includes GeoLite data created by MaxMind, available from
791http://www.maxmind.com/.
Note: See TracBrowser for help on using the browser.