source: trunk/changelog.txt

Last change on this file was 235, checked in by bowman, 11 years ago

Various untested changes: experimental catch-all mode, alternate cache overwrite handling

File size: 94.9 KB
Line 
1Cardservproxy changelog:
2------------------------
3
40.9.1 - RC4
5
6- Added: An experimental catch-all mode, that allows for unidentified traffic (unknown onid/caid or matching no profile).
7 Messages in this category with no onid will receive a dummy value (fffe). The mode is enabled either by adding at
8 least one chameleon-connector with catch-all set to true, or by setting catch-all true for the extended-newcamd port.
9 If enabled, block-caid-mismatch defaults to false (so no need to set it), and most warnings in the extended port and
10 the chameleon connector(s) will be hidden. This also changes cache-behavior to always wait the full max-cache-wait
11 before moving on to connector forward.
12 The dummy catch-all profile uses the global max-cw-wait set for the connector-manager (default 9000ms).
13- Added: Overwrites in the cache will now retain both cw candidates (up to 3 extra will be kept for a request). Unique
14 new candidates will be counted (new cache stat counter "contested" added).
15- Fixed: Handling for unknown sid (0 or a dummy sid listed in dummy-services) was broken in 0.9.0.
16- Fixed: Incoming requests for dummy sids should not retain the dummy when forwarded, now changed to 0.
17- Fixed: Per profile max-cw-wait could not be set below 1s (now both per profile and global allow down to 100 ms).
18 NOTE: While it may now be possible to configure such low values, whether it works or not below 1s is still undefined.
19- Fixed: Probing was sometimes attempted even when there was only one candidate connector.
20- Fixed: Radegast support updated to make sense with 0.9.x.
21- Fixed: Emm handling for extended newcamd in systems with signifcant provider-idents.
22- Fixed: Now possible to move messages between profiles from plugins, changes to networkid/caid were previously ignored.
23- Fixed: Per user sid lists provided by custom usermanagers were ignored for multi-context sessions (Csp, ExtNewcamd).
24- Fixed: The default-deny-list wasn't checked for CspConnect connection attempts.
25- Fixed: Closing of old sessions (on new login) wasn't working for ExtNewcamd.
26- Fixed: Debug mode for the "*" pseudo profile (Csp, ExtNewcamd) couldn't be turned off via config (attribute ignored).
27- Fixed: Enigma1/2 bouquet file generator in status web now works for multi-profile setups. Additionally, it will
28 include services for which no mapping data exist (unscrambled, or never watched). Still requires enigma service files.
29- Fixed: Using manual service mapping (can-decode list) for one connector would affect discovery of the listed services
30 for other connectors in the same profile.
31- Fixed: It was not possible to specify manual service mapping lists for csp and chameleon connectors, for more than
32 one profile at a time.
33- Fixed: Removing a profile from a user that had used csp-connect had no effect, as it remained locally cached at the
34 client proxy, and no mechanism existed for removing it or blocking the now disallowed requests.
35- Fixed: Missing log warning and cws event for csp-connector login failure (on bad credentials).
36- Fixed: Massive memory leaks when using high volumes of ClusteredCache udp updates in broken/assymetric clusters, or
37 when using third party relayers (probably still a bad idea to do this however).
38- Changed: Two ecms (or dcw replies) with the same payload data but different dvb table ids (even/0x80 vs odd/0x81) are
39 now considered identical by the proxy. Implications unknown.
40- Changed: ClusteredCache transport format now includes meta-data such as ca-id and network-id.
41 NOTE: This breaks compatibility with previous versions. If one node has the new format, all nodes must be upgraded.
42 The source ip's of any received incompatible updates are listed in the property 'version-mismatch' (when possible).
43- Changed: Enabling debug for a user now also enables web transaction logs for that user (same as profile debug).
44 NOTE: Kicking the users sessions may be required before this takes effect (both after enabling and disabling).
45- Added: Experimental resend/broadcast mechanism for ClusteredCache, to make cache-only nodes slightly more feasible.
46 This allows a node to ask all peers for a cw that doesn't exist locally and is taking more than max-cache-wait / 2.
47- Added: Option to automatically add any peers that are sending updates to ClusteredCache.
48- Added: Cws event for found service now shown in the status-web (despite the command being called error-log).
49- Added: Experimental sid cache linking feature, allows cache hits based on meta-data like sid rather than ecm payload.
50 Links added via the web commands are stored in etc/links.cfg (this file is monitored for changes and autoloaded also).
51- Added: More sanity checks to DcwFilterPlugin (checksums, length and an option to monitor all replies for duplicates).
52- Added: DreamboxPlugin (csp-agent) support for more/older dm500 images. Basic (unsecured) file upload support.
53- Added: proxy-reference.html updated with highlighting for important elements and additional examples.
54- Added: CacheCoveragePlugin, for visualizing the contents and show coverage per transponder/mux.
55- Added: New AdvXmlUserManager to show how to add um-specific functionality without changing the core or existing um's.
56- Added: Some basic sanity checks for incoming ClusteredCache udp updates (DCW checksum, zero counting). Default enabled.
57
58- Changes to proxy.xml:
59 Added: Element <auto-add-peers> to ClusteredCache <cache-config> (true/false, default: false). Add peers automatically
60 if they send you traffic (adding them as targets for outgoing packets).
61 Added: Element <cw-validation> to ClusteredCache <cache-config>. Attributes 'checksum', 'zero-counting', 'log-warnings'
62 (true/false, default true). Determines whether to drop remote-origin cw's with bad checksum, and more than 4 zeroes
63 (but less than 8, so intentionally blanked out cw's will not be affected by this check).
64 Added: Attribute 'enable-service-linking' to <cache-handler> (true/false, default: false). Enables sid cache linker.
65 Added: Attribute 'allow-different-ip' to <user-manager>, allows newcamd connections from different ip sources
66 for the same user.
67 Added: Element <hide-radio-services> to <mapper> (true/false, default: true). Show/hide radio services in web/xml api.
68 Added: Element <catch-all> to <chameleon-connector> (true/false, default: false). Ignore remote meta-data and forward
69 anything to this connector, regardless of properties.
70 Added: Attribute 'catch-all' to <extended-newcamd> (true/false, default: false). Allow unidentified traffic (that
71 doesn't match any profile). Only makes sense with a perfect cache, or a chameleon-connector with catch-all.
72
73- Changes to the http/xml api:
74 Added: Attribute 'filtered-by' to <ecm> (transaction logs). Contains the reason for a B flag, or the filtering plugin.
75 Added: New event type 9 - "found service" added to connector events output (error-log, yes - not really an error).
76 Added: New ctrl command 'set-connector-metric', allows changing individual connector metrics without config reload.
77 Added: New ctrl command 'set-au-user', forces a specified user to au status for a connector (kicking existing sessions)
78 Added: Attributes 'reset-services', 'blocked-services', 'allowed-services' to <profile> (ca-profiles output). Shows
79 the currently configured lists resolved with service names (but only if they're shorter than 25 entries).
80 Added: Attribute 'include-parsed' to <all-services>. Includes services with no mapping data (info parsed from file only).
81
82
830.9.0 - 2010-08-15 (see README.0.9.0.txt for conceptual changes and tips)
84
85- Fixed: The included ConaxConnector plugin in 0.8.13 was an older version.
86- Fixed: Extra http auth login was required for accessing plugin webs (bug introduced in 0.8.13).
87- Fixed: Probing of connectors with unknown status wasn't done when cache hits occured.
88- Fixed: No longer possible to create multiple profiles where both ca-id and network-id are the same.
89- Fixed: Anonymized (non-au) newcamd card-data can no longer end up with user id 1 (this confused some clients).
90- Fixed: Disabling a profile now automatically disables any connectors that explicitly references it.
91- Fixed: Enigma services file parsing now uses comma separated filter strings, to allow names with spaces.
92- Fixed: No longer possible to set the same user as au-user for multiple cards within the same profile.
93- Fixed: Initial web startup is now delayed until after there is a usermanager available.
94- Fixed: CCcam.channelinfo parsing errors (some of them).
95- Fixed: Potentially serious and long standing ClusteredCache bug that caused sporadic io errors when sync-period > 0.
96- Changed: Now possible to configure max-cache-wait both as a fixed time value (as before) and as a percentage of
97 the max-cw-wait for the request. I.e if set to the string "50%", requests for a profile with 9000 ms max-cw-wait would
98 end up with a max-cache-wait of 4500 ms, while requests for another profile with 650 ms would get 325 ms.
99- Changed: Adding manual can-decode/cannot-decode services to a connector will now also update any previously
100 automapped data to prevent conflicting/overlapping information (so no need to clear maps after manual changes).
101- Changed: Network-id is now used in enigma services file parsing only when no provider string filter is set.
102- Changed: All time fields in the config can now be specified in minutes/secs/millisecs by adding a suffix (m, s, ms).
103 If no suffix is added, the old default for the field will be assumed (so configs/docs remain compatible).
104- Changed: Added and updated defaults in the generated proxy.xml template to make more sense with the current version.
105- Changed: ClusteredCache sync-period is now used even in receive-only mode (without peers). This can allow you to
106 significantly increase cache hits at the expense of ecm transaction time. ClusteredCache is now used by default.
107- Added: Services parser for dvbviewer exports (ini files). The filter string is matched against the "Root" key if set.
108- Added: Services parser for neutrino services.xml.
109- Added: Timed ecm blacklist per connector, to avoid forwarding the same ecm several times to a connector that can't
110 handle it (mainly when there is no sid to go by in the request). Entries will be kept for 3*max-cw-wait.
111- Added: Slightly better awareness of satellite concepts like provider-idents and other ca-system-specific artefacts.
112 This includes extending the service mapper with an additional custom-id/cid (besides sid) for systems like irdeto.
113 For some systems this will include provider-ident as a factor in the mapping (with require-provider-match for profile).
114- Changed: Max-connections changed to a per-profile value, to handle the satellite scenario of the same user connecting
115 to multiple profiles. I.e now max-connections 1 means the user is allowed 1 connection in every profile they have.
116 NOTE: This means if you change the value or add ports to a profile, you may have to kick users before it takes effect.
117
118- Added: New connector type 'csp-connector' specifically for chaining multiple proxies together. Requires that the
119 ONID (network-id) and ca-id is set properly for all profiles in all involved proxies. This type allows multiple
120 profiles to be shared over a single connection, and prevents loops (forwarding the same ecm back and forth between
121 proxies that have each other as connectors).
122 The protocol is documented in the source and connections are initiated using the httpd (so ssl can/should be used).
123 See proxy-reference.html for more info.
124- Added: New connector type 'chameleon-connector' for connecting to newcs as mgcamd and accessing multiple cards in one
125 newcamd session. Only properly identified traffic can be sent to this connector type (known caid + provider ident).
126 Only remote cards that map into locally defined profiles (matching caid/provider ident) will be used.
127- Added: Support for mgcamd/newcs newcamd-extensions in incoming connections (via a single extended-port for all
128 profiles), using multiple systems over a single newcamd session. For this to work all combinations of caid and
129 provider ident must map to a profile with network-id set. Ambiguous traffic will be denied.
130- Added: Redundant forwarding. The service mappers can now be configured to select two of the least loaded connectors
131 instead of just one (if two or more candidate connectors exist for a request). If enabled, this can up to double the
132 load on the cards, but assuming enough capacity exists it will mean always having a backup ready in case the primary
133 connector choice failed/timed out for any reason. Should improve reliability in single-node proxy setups.
134- Added: Plugin dependency resolver. This makes it easy to build plugins that make use of existing 3rd party libraries,
135 by fetching jars automatically on first load. See README.Plugins.txt for more info and DreamboxPlugin for examples.
136- Added: getProperties() method in the plugin interface, for returning arbitrary usage information shown in the output
137 of the proxy-plugins status command.
138
139- Added: When using only asynchronous connectors, it is now possible for a client session to get a cache hit even after
140 a forward to card was initiated. This can result in transactions with both F and C/R flags.
141- Added: Last-seen data now also contains entries for failed login/connect attempts, available through a new status
142 command 'login-failures' (available to all users, but non-admins can only see attempts made with their user name).
143- Added: New interface ReplyFilter that plugins can use to intercept and alter/block DCW's as they're returned from
144 connectors, before they're processed by the proxy (possibly find and delete bad CW's). See README.Plugins.txt.
145 A DcwFilterPlugin that illustrates this and blocks some common bad responses is included.
146- Added: Fixes for running under jamvm on embedded systems (including the broken auto-generating of the config template).
147- Added: New status commands for troubleshooting: 'export-services', 'system-properties', 'system-threads', 'file-log'
148 and 'proxy-plugins'.
149- Added: Option to configure the date-format used by the default logger (allows easier fail2ban monitoring).
150- Added: Arbitrary meta-data/remote info can now be returned by connectors (for display/troubleshooting/statistics).
151- Added: Multiple client ids (oscam, scam, rq-sssp etc).
152- Added: More example plugins included (and updated versions of the rest).
153- Added: JVM version check. The proxy will refuse to start with anything but a sun jvm. If you're absolutely sure, this
154 can be overridden by adding the following to the java cmd line: -Dcom.bowman.cardserv.allowanyjvm=true
155
156- Changes to proxy.xml:
157 Added: Element <csp-connect> to <status-web> (to receive csp-connections, enabled by default).
158 Added: Element <csp-connector> to <connectors> (to define csp-connectors). See proxy-reference.html.
159 Added: Element <chameleon-connector> to <connectors> (connector to a newcs/chameleon setup as mgcamd). Same as a
160 newcamd-connector, except it is not bound to a profile, always asynchronous and ignoring the client-id setting.
161 Added: Element <extended-newcamd> to <profiles> (unbound port for extended newcamd protocol, as used between mgcamd
162 and newcs). Allows mgcamd to use multiple systems (all profiles the user has access to) over a single connection.
163 Added: Element <log-dateformat> to <logging> (optional java SimpleDateFormat string to use for the standard logs).
164 Added: Attribute 'provider-idents' to <profile> (optional, allows listing of provider-idents, even with no connectors).
165 Added: Attribute 'require-provider-match' to <profile> (true/false, default: true). Set to false if you know that for
166 this profile, provider idents in ecm requests do not need to match those on the cards (this is the case for irdeto).
167 NOTE: If require-provider-match is false, provider-idents will get 000000 added automatically. Conversely, if only
168 ident 000000 is specified for a profile, require-provider-match defaults to false instead of true when omitted.
169 Added: Attribute 'provider-idents' to <newcamd-connector>, (optional, overrides the idents from the server/card).
170 Added: Attribute 'exclusive' to <can-decode-services> (true/false, default: false). Set to true for a list to indicate
171 that there should be no probing done for the connector, only those services listed are to be considered decodable.
172 Added: Attribute 'profile' to <can-decode-services> and <cannot-decode-services>. Only applicable for connector types
173 csp-connector and chameleon-connector, where multiple lists can be used to specify services for several profiles.
174 that there should be no probing done for the connector, only those services listed are to be considered decodable.
175 Changed: Attribute 'provider' for <service-file> changed to 'filter' to avoid confusion. If provider-idents have been
176 specified correctly for the profile, there is no longer any need to repeat that list in the case of cccam parsing.
177 Changed: All elements that allowed hex sid lists to be specified (per connector or profile) now accept an alternate
178 syntax sid:cid (where cid is custom id, used for situations like the irdeto chid where sid alone is not enough).
179 NOTE: <allow-services> is an exception, checks against that list are made with sid only.
180 Changed: Element <unknown-sid> removed from <mapper>, replaced with <dummy-services> that may contain multiple sids.
181 Use this to list any fake sids used by limited clients that can't know the real one, to avoid interference with maps.
182 Added: Element <redundant-forwarding> to <mapper> (true/false, default: false). Can be set globally or per profile,
183 as with other mapping settings. Transactions that trigger redundant forwarding will get the new flag '2'.
184 Added: Attribute 'include-file-events' to <warning-threshold> (true/false, default: true). Setting this to false
185 disables the 'file-log' http query (no file log events will be intercepted for display on the web).
186
187- Changes to the http/xml api: (always use /xmlHandler?command=status-commands or ctrl-commands to see syntax).
188 Added: New status command 'export-services', dumps the internal state of the service maps (admin only). Add the param
189 format=hex for an alternate format matching the sid lists used in the config.
190 Added: New status command 'system-properties', shows the JVM system properties (superuser only).
191 Added: New status command 'system-threads', dumps all JVM threads as strings (superuser only).
192 Added: New status command 'login-failures', shows a list of failed login attempts per user or ip (for most interfaces).
193 Added: New status command 'proxy-plugins' for listing all loaded plugins and any associated info they publish.
194 Added: New status command 'file-log', returns recently intercepted file loggings with level WARNING or SEVERE.
195 Added: New ctrl command 'gen-keystore', auto creates a java keystore for using the status web with SSL.
196 Added: New ctrl command 'disable-connector', temporarily disables a specified connector.
197 Added: New ctrl command 'set-profile-debug', temporarily changes debug flags (set to false for ALL to delete ecm logs).
198 Added: New ctrl command 'set-user-debug', temporarily changes debug for a user (enabling log-ecm, log-emm, log-zap).
199 Added: New ctrl command 'remove-failed', removes entries matching specified wildcard mask from login-failures.
200 Added: New ctrl command 'clear-file-log', removes intercepted file log entries from the web-backend.
201 Added: Attribute 'time' to <jvm> (proxy-status output). Local system time as a rfc822 date.
202 Added: Attribute 'cdata' to <service> (most output containing services). Custom data for service mapping (chid/ident).
203 Added: Element <remote-info> to <connector> (cws-connectors output). List of <cws-param> elements with name/value
204 attributes, containing arbitrary information about the connector.
205 Added: Attributes 'network-id', 'ca-id', 'provider-ident' and 'origin-id' to <ecm> (transaction logs). These are only
206 included when the transaction occured in the '*' profile, and origin-id only for CspSession transactions.
207 Added: Attribute 'au' to <session> for NewcamdSessions. Indicates which connector the session is forwarding emms to.
208 Added: Attribute 'build' to <proxy-status>. The build number for the running cardservproxy.jar.
209
210
2110.8.13 - 2009-10-18
212
213- Added: New attribute per profile 'network-id' (the original dvb network id, 4 hex digits). This will uniquely
214 identify profiles between proxies, instead of ca-id. It will also be used in filtering enigma services files.
215 See proxy-reference.html for info about how to find the id if you don't know your dvb-network.
216- Added: Optional mapping table to auto-assign newcamd connectors to profiles based on ca-id, to make this work as
217 before even when multiple profiles use the same ca-id (no longer any need to set 0000, use the correct id always).
218 Use this only when it is unknown which card a remote newcamd server contains (or when this changes randomly).
219- Added: ClusteredCache now avoids locking for requests where a remotely received cache lock has the same ip as
220 requesting client, to deal with situations where both connectors and cache sharing are used to link multiple proxies
221 together (prevents incoming ecm requests from a remote proxy being blocked by the cache lock from that same proxy).
222- Added: Plugins can now affect connector selection, if they implement the interface CwsSelector. They'll be given a
223 chance to exclude connectors from the selection process for each message.
224- Added: Cache hits where the cached request had a different ca-id will now be blocked, to avoid misleading clients
225 that are sending requests to the wrong profile. This can be disabled using new global setting <block-caid-mismatch>.
226- Added: Disconnected connectors now show when they went offline.
227- Added: Newcamd OSD messages are now sent to Acamd as well (previously only to Mgcamd).
228- Added: More user events in the remote api (login, logout, login failure). Plugins can use these as well.
229- Changed: ClusteredCache now uses a much more compact format to reduce bandwidth (not compatible with old versions).
230- Fixed: ClusteredCache bandwidth usage estimates are now properly calculated even with multiple peers and sync-period.
231- Fixed: Minor browser-specific issues with the status web layouts (clear the browser cache or force a reload).
232
233- Changes to proxy.xml:
234 Added: Attribute 'network-id' to <profile> (4 hex digits, i.e "a027"). Set this to the original dvb network id.
235 Added: Element <block-caid-mismatch> (true/false, default: true) to section ca-profiles.
236 Added: Element <caid-profile-map> to <newcamd-connector> (caid=profilename, entries separated by space). This element
237 is mandatory if no profile attribute is set for the connector.
238 For example: <caid-profile-map>0b00=cable 0500=terrestrial</caid-profile-map)
239
240- Changes to the http/xml api:
241 Added: Attribute 'disconnected' to <connector> (cws-connectors output). The time of last disconnect (if disconnected).
242 Added: Attribute 'network-id' to <profile> (ca-profiles output).
243
244
2450.8.12 - 2009-06-29
246
247- Fixed: No forwarding attempt will now be made if a cache timeout meant that max-cw-wait was already exceeded.
248- Added: Experimental broadcasting of ecms without sid to all non-congested connectors in profile (off by default).
249- Added: Client id for DiabloCam wifi (uw?).
250- Added: Command to clean the last-seen log.
251- Added: Made the connector reconnect attempts more visible and consistent in the status web.
252- Changed: Radegast sessions are no longer added to disconnected users/last-seen log.
253- Changed: Connectors that fail on proxy startup are now added anyway, to make them visible via the http/xml api.
254 NOTE: If the failed connector has no profile specified, it will still not be visible until it has connected properly.
255- Fixed: Error when unloading the LoggingPlugin.
256- Fixed: Lost service warnings are no longer shown for sids that have been made sticky using can-decode-services.
257- Fixed: Slow memory leak in ClusteredCache when using sync-period.
258- Fixed: Typos in the generated proxy.xml template.
259- Fixed: Inconsistent username case handling.
260- Fixed: Card-data with extra trailing bytes would break sessions for non-au users (rqcs).
261- Fixed: The open-access handling should now work for xmlusermanager as well as simpleusermanager.
262- Updated ConaxConnector to latest version.
263
264- Changes to proxy.xml:
265 Added: Element <hide-disabled-connectors> (true/false, default: false) to section mapper.
266 Added: Element <broadcast-missing-sid> (true/false, default: false) to section mapper.
267
268- Changes to the http/xml api:
269 Added: New ctrl command 'remove-seen', removes users matching the 'name' parameter from the last-seen log.
270 Added: Attribute 'next-attempt' to <connector> (cws-connectors output). Shows the nr of secs until next retry.
271
272
2730.8.11 - 2009-05-01
274
275- Added: Jvm stats in the debug logging: heap used/allocated, thread count, file descriptors (used and max, unix only).
276- Added: UserManager interface now has a getDisplayName() method, to allow for an optional display-name attribute.
277- Added: Client id for Rq-echo-client and sbcl.
278- Added: Config and Admin sections (status-web) are now disabled by default. List users that should have access to
279 these in the new super-users element in the status-web section.
280- Added: MessagingPlugin can now send automated email with the same type of triggers as for mgcamd-osd.
281- Added: Msg filtering now shows up in LoggingPlugin (assuming the filtering plugin used msg.setFilteredBy() to do it).
282- Added: SimpleUserManager now allows for open access, accepting any newcamd connections as long as they have the right
283 (common) password. Open access can be restricted to specific profiles or to usernames starting with a certain prefix.
284 NOTE: Random user names will be assigned, but the display-name will be whatever the client specified.
285- Added: Made it possible to configure card-data (non-au) for newcamd ports without editing the card-files manually.
286- Added: A parser for using CCcam.channelinfo as the services-file for a profile. As with enigma files you need to
287 specify which sids are relevant for this profile by creating a filter. See proxy-reference.html for details.
288- Added: Externally loaded plugins can now be individually re-loaded just by replacing the jar file.
289- Fixed: When using the status-web without ssl, any specified bind-ip was ignored.
290- Fixed: Distribution tars finally corrected.
291- Fixed: Enigma bouquet-file generator in the status web might actually produce valid files now (enigma services-file).
292- Fixed: Status-web session transfer from script/ajax context to browser context via cookie was always broken.
293- Fixed: Made card-data parsing less sensetive to errors/truncated data.
294- Changed: More minor tweaks of the status web views.
295- Changed: Blowfish encryption tool trtest.jar renamed to: fishenc.jar
296- Updated the codemirror xml/js editor (used in the status-web) to version 0.61.
297
298- Changes to proxy.xml:
299 Added: Element <super-users> to <status-web>. List of user names that should have access to the Config and Admin
300 sections (the users must have admin="true" or they'll be ignored).
301 Added: Element <open-access> to <auth-config> (SimpleUserManager/XmlUserManager). See proxy-reference.html.
302 Added: Attribute 'ca-id' to <card-data> (newcamd listen ports). The ca-id returned to clients, when type="config".
303 Added: Element <providers> to <card-data>. List of providers returned to clients when type="config". 3 bytes each,
304 separated by comma, i.e: 00 00 00,00 00 01,00 00 02
305
306- Changes to the http/xml api:
307 Added: Attributes 'filedesc-open' and 'filedesc-max' to <jvm> (proxy-status output). Used/max file-descriptors, only
308 available on unix jvm's (java6+). See README.Optimization.txt for info on increasing this limit on linux.
309 Added: Attribute 'super-user' to <status> (cws-login output for successful login). Indicates whether the user has
310 access to control commands and is allowed to modify the config.
311 Added: Attribute 'display-name' to <user> (proxy-users output). The status web will use this primarily, if available.
312
313
3140.8.10 - 2009-03-08
315
316- Fixed: The experimental strict synchronization for ClusteredCache now does what it claimed to (sync-period).
317- Fixed: Setting debug="true" for the logging now outputs any stacktraces at WARNING level, as indicated in the docs.
318- Fixed: Sessions weren't always being properly disconnected when session timeout was hit (introduced in 0.8.9).
319- Fixed: Newcamd sequence nr wasn't set correctly in EMM replies (caused some clients to log errors).
320- Fixed: Possible socket handle leak when new sessions couldn't be created. This may have caused listen ports to close.
321- Fixed: The default value for user attribute max-connections is now calculated correctly, based on the total number of
322 newcamd listen-ports in the profiles the user has access to. See INFO logging on startup and on connect for the values.
323- Fixed: If a user exceeds the max-connections count, the session that is closed is now always the oldest one.
324- Changed: Slightly improved views and more stat values in the example status web.
325- Changed: ClusteredCache now accepts multiple proxies running on the same host name (on different ports).
326- Changed: ClusteredCache is now slightly less inefficient with its bandwidth usage (still much room for improvement).
327- Added: ClusteredCache now pings each peer regularly to keep track of latency (breaks compatibility with old versions).
328- Added: ClusteredCache additional stat counters (toggle debug for the cache to see all).
329- Added: Custom usermanager and cache implementations can now be loaded with the plugin classloader (from a jar-file).
330- Added: Additional custom connector implementations can now be loaded, also via the plugin classloader if desired.
331 A new example connector is included with the plugis: ConaxConnector - It reads local conax cards directly using java6
332 and pcsc card-readers. See README.ConaxConnector.txt for more info.
333
334- Changes to proxy.xml:
335 Added: Attribute 'jar-file' to elements <user-manager> and <cache-handler> (optional). If this is specified, the class
336 will be loaded by the plugin classloader from the named jar-file (path relative to the plugins dir).
337 NOTE: This doesn't mean the implementations can be re-loaded dynamically, restart is still required for new jars.
338 Added: It is now possible to specify custom connector implementations using the same notation as the built in newcamd
339 and radegast connectors, but with the added attributes of 'class' and optionally also 'jar-file'.
340 If a jar-file is specified the plugin classloader is used, as above. For example:
341 <conax-connector name="lcard" profile="sat2" class="com.bowman.cardserv.cws.ConaxCwsConnector" jar-file="conaxconnector.jar">
342 <!-- config goes here -->
343 </conax-connector>
344
345- Changes to the http/xml api:
346 Added: Attributes 'active' and 'keepalive-count' to the <session> element (proxy-users output). This is used to show
347 which inactive/idle sessions are sending keepalives in the status web (they are shown in blue). All inactive rows
348 will show with italic font.
349 Added: Element <listen-port> to <profile> (ca-profiles output). Listen ports are now separate child elements with the
350 attributes: name, protocol, port-number, alive (true/false), properties (custom settings as key=value string).
351 Removed: Attribute 'listen-ports' from <profile>, replaced with the above list of <listen-port> elements.
352
353
3540.8.9 - 2008-11-04
355
356- Fixed: Possible infinite loop state for the file-change-watcher (would fill sysout log once triggered).
357- Added: The reset-connector ctrl command can now be used to delete all mappings (full reset).
358- Added: Client ids for cardlink.nl and octagon stbs.
359- Added: A test-delay feature for the LoggingPlugin. This allows for a manually added delay applied to all logged
360 requests (before they are processed). The delay can be set via status web command, and is intended as tool for
361 finding the exact freeze-time for a given ca-profile, i.e: gradually increase the delay until freeze and note the
362 full ecm transaction time (roundtrip) in the client logs. The LoggingPlugin has also received a feature for sending
363 arbitrary newcamd-messages. This can be used to explore the capabilities of different clients in realtime.
364- Added: New setting for allow-services lists, per profile (inverse of block-services).
365- Added: The duration column in the status-web now shows the time since last zap (if any has occured since connect).
366- Added: Feature for sending keep-alives to clients as well as servers. Normally only the client sends these in newcamd,
367 but most clients seem to ignore incoming keep-alives. It can be used to find dead sessions faster in very large shares.
368- Changed: XmlUserManager now considers deleted users as disabled (causing any active sessions to be kicked).
369- Changed: max-cw-wait can now be set per ca-profile, as it is typically different from one ca-system to the next.
370 This also means all capacity estimates (status-web) can be made more accurate. Use the test-delay feature of the
371 LoggingPlugin to find the freeze-time for each profile, and set the max-cw-wait to this (or 1 second above).
372- Changed: Moved more previously hard-coded settings to config. No need to touch these unless you know what its about.
373- Updated the codemirror xml/js editor (used in the status-web) to version 0.58.
374- Misc minor fixes.
375
376- Changes to proxy.xml:
377 Added: Element <max-threads> to <ca-profiles> (default: 1000). If this is reached the proxy will stop accepting
378 connections until it drops again. The fixed default in 0.8.8 was 500.
379 Added: Element <session-timeout> to <ca-profiles> (default: 120, in minutes). Maximum idle time for user sessions.
380 Added: Element <newcamd-maxmsgsize> to <ca-profiles> (default: 400). This is CWS_NETMSGSIZE. The old default was 240,
381 so try that if you run into any problems related to message size.
382 Added: Element <allow-services> to section mapper. List of sids (hex). The inverse of <block-services>, only sids
383 listed here will be passed through to connectors. This can help filter out requests with bad ecms, and reduce
384 unecessary probing of the cards. As for block-services, it only makes sense to specify this element per profile.
385 Added: Elements <max-cw-wait> and <congestion-limit> to <profile>. Same settings as for the connection-manager, but
386 allows overriding the global setting per profile.
387 Added: Element <session-keepalive> to <ca-profiles> (default: 0, in minutes = off). Sends keep-alives to clients.
388 Attributes are: exclude-clients (list of client names, default: "" = send to all). Exclude listed client types.
389
390- Changes to the http/xml api:
391 Added: Attribute 'id' to the <session> element (proxy-users output). The numerical id for the session, can be matched
392 to the proxy logs (or LoggingPlugin log file names).
393 Added: Attribute 'idle-time' to the <session> element. Shows how long the connection has been idle.
394
395
3960.8.8 - 2008-09-23
397
398- Added: Client id for acamd.
399- Added: Support for multiple xml sources in XmlUserManager (see README.XmlUserManager.txt). This also fixes handling
400 of deleted users (which previously required a restart/config reload).
401- Added: Support for enigma2 format in bouquet file generator (status web channels section).
402- Added: Various mechanisms to protect the proxy from connector disconnects in case of misbehaving/buggy clients.
403- Added: Multiple format support for the services file importer (assigning names to sids). At the moment only two:
404 enigma - the default (as before, enigma1/2 services file with optional provider string filter)
405 simple - a plain text list with hex-sid=service name (provider attribute is ignored, use one file per profile)
406- Changed: Now skipping all host name/cert verification when making outgoing connections to https urls.
407- Changed: LoggingPlugin now saves files named with session id instead of ip (separates multiple sessions from one ip).
408- Changed: Moved MAX_Q_SIZE and MIN_DELAY to proxy.xml to allow for additional tweaking of the connector behavior.
409 Don't touch these unless you fully understand the implications.
410- Changed: The manual service mapping (<can-decode-services>/<cannot-decode-services> per connector) is now profile
411 specific rather than global. This matches the automatic service mapping and will hopefully cause less confusion.
412- Changed: Clicking on the user names in the status web sessions view now links to the full xml for that user. The ecm
413 transaction log is available via clicking on the ecm count instead (last-seen also changed to match this).
414 Additionally, if a user session currently has more than 1 pending ecm this will be shown in (red) in the Iv column.
415- Fixed: Minor leaks related to logging.
416- Fixed: Rare date formatting errors (mainly for the status web).
417
418- Changes to proxy.xml:
419 Added: Element <default-max-queue> to <connection-manager> (default: 50). Max queue length allowed to build up on
420 one connector before the proxy assumes something has crashed and disconnects it.
421 Added: Element <default-min-delay> to <connection-manager> (default: 10, in ms). Delay inserted between consecutive
422 ecms to one connector in async mode. Workaround for servers that misbehave when requests are too close together.
423 Added: Elements <max-queue> and <min-delay> to <newcamd-connector>/<radegast-connector>, same as above but specified
424 per individual connector (allowing different values for different connectors).
425 Added: Attribute 'format' to <services-file> (default: enigma). Example for the "simple" format:
426 <services-file format="simple">etc/services.properties</services-file> <!-- format e.g: 03fb=Service Name -->
427
428- Changes to the http/xml api:
429 Added: Attribute 'last-zap' and 'pending-count' to the <session> element (proxy-users output). Indicating the time
430 since last 'Z' flag and the number of currently pending requests a session has (>1 means its using async mode).
431 Added: Attribute 'context' to the <session> element (proxy-users output). This shows the context of the last ecm
432 from the client (roughly = the card that the client believes it is connected to).
433 Added: Attribute 'unknown-newcamd' to <ecm> elements (user/cws-log output). This is an attempt to chart what the
434 various clients and servers use the "undocumented" extra newcamd bytes for: offsets 4 - 9, and the 4 upper bits of
435 offset 11. The attribute will contain these values for both the request (>) and the reply (<).
436
437
4380.8.7 - 2008-08-23
439
440- Added: Connector info now contains details about the received newcamd card-data (status web/xml api).
441- Added: Client id for rqcamd.
442- Fixed: Capacity estimates remained for cards that were disconnected, they are now excluded from the totals.
443- Fixed: Plugins only worked if all of them were externally loaded (from separate jar file). I.e: LoggingPlugin failed.
444- Fixed: A few unlikely memory leaks related to connectors ending up in a zombie state (seemingly ok but locked).
445- Changed: Only client id 0000 is now displayed as Generic, other unknowns will be shown by the actual numerical id.
446- Changed: Transaction flags are now in the order in which they are set internally by csp, not alphabetical.
447- Minor updates to the example plugins.
448
449- Changes to the http/xml api:
450 Added: Attribute 'card-data1' and 'card-data2' to the <connector> element (cws-connectors output). Admin only.
451
452
4530.8.6 - 2008-07-27
454
455- Changed: Xml status commands are now handled like the control commands, they can be registered and added on the fly
456 by any user component. This also makes it possible to override the default command handlers with your own.
457- Changed: Reworked the "filters" interface to a more generic plugin framework, to enable quick extensions that
458 are not directly related to the ecm traffic (LoggingFilter changed to LoggingPlugin).
459- Added: GeoipPlugin. Illustrates the plugin framework and command overriding (adds a google maps + geoip mashup).
460- Added: EmmAnalyzerPlugin. Gathers statistics regarding emms received from clients.
461- Added: MessagingPlugin. Auto mgcamd-osd replies for client session (e.g "service unavailable") and mail to users.
462- Added: New flag '1' (one), assigned to the first transaction a session performs (instead of Z as before).
463- Added: Stat counter for denied/blocked ecm's (flag N).
464- Added: Missing client ids (cccam, evocamd, alexcs etc).
465
466- Changes to proxy.xml:
467 Changed: Elements <proxy-filters>, <filter> and <filter-config> renamed to <proxy-plugins>, <plugin> and
468 <plugin-config>.
469 Added: Attribute 'jar-file' to <proxy-plugin>. Allows loading the plugin using a separate classloader, from an
470 external jar file in the "plugins" dir. This way plugins can be replaced/reloaded at runtime without restarting
471 the proxy (they are reloaded and restarted each time the proxy.xml config is touched/updated).
472 Added: Element <delay-missing-sid> to <connection-manager> (default: 100, in ms). This adds a short delay for any
473 incoming request without sid, before the cache is checked. The idea is to increase the likelyhood of a another
474 request for the same ecm (but with sid specified) arriving first in the cache. This is to avoid having a large
475 number of clients waiting in the cache for a forward that might get routed to the wrong card (because it had no sid).
476
477- Changes to the http/xml api:
478 Changed: Status command 'error-log' will now always honor a profile selection (previously admin users would get all
479 profiles regardless of preference).
480 Added: New status command 'status-commands', lists meta-data for all registered status commands.
481
482
4830.8.5 - 2008-06-29
484
485- Fixed: Proxy now sets correct service id in cache hit replies (i.e same as in the request, unless 0).
486- Fixed: Repeated web events for "cws connected" retries when the server returned invalid card data (caid 0 or mismatch).
487- Fixed: Connector name was sometimes missing in the transaction time breakdown even though the F-flag was set.
488- Fixed: XmlUserManager will now keep its last known working set of users when proxy configuration changes are made.
489- If a reply was received from a remote cache, the connector name in the transaction data is now prefixed with:
490 "remote:" - to make it possible to distinguish when the same names are present locally.
491- "Last seen" session data now includes last known ip address for the user.
492- Cache flag 'O' is now strictly for timeouts in the cache (max-cache-wait exceeded). Added new flag 'Q' for aborts due
493 to forward failures (remote or local).
494- Added filtering per profile to status web (only shown for users with access to more than one, in events + channels).
495- Added ecm load estimates to status web totals and ca-profiles (sums based on the cws-connector data below).
496- Added a tool for tracing configuration file use in the proxy. Set -Dcom.bowman.cardserv.util.tracexmlcfg=true when
497 starting the proxy and all config access will be traced. This trace can be written to file (etc/xmlcfg.txt) by using
498 the CtrlCommand "dump" from the admin page of the status web (or by using the http/xml api directly).
499
500- Changes to proxy.xml:
501 Added: Element <hide-names> for ClusteredCache (true/false, default: false). Set to true to stop the cache from
502 sending the connector names to the remote targets (only makes sense if they are untrusted/unknown and the names
503 contain sensetive information).
504 Changed: Attribute 'debug' for <profile> now defaults to true. The flag is only used to enable the transaction
505 backlog and there is typically no reason not to have that.
506
507- Changes to the http/xml api:
508 Added: Attributes 'request-hash', 'cw' and 'warning' to the <ecm> element (user-log output). The request hash allows
509 comparison with the hash values logged elsewhere, and makes it possible to identify a particular ecm request.
510 If 'cw' is not present, it indicates the client received an empty reply (flag 'E' should also be present).
511 If 'warning' is true, the proxy considered this transaction a problem and logged it also to the user-warning-log.
512 Added: Attribute 'host' to <entry> (last-seen output).
513 Fixed: Status command 'ctrl-commands' was available for non-admin users via http GET (listing command definitions
514 only, no actual execution possible).
515 Added: New status command: cws-log (params: name). Allows the admin user to view the last 100 transactions for the
516 specified cws-connector. Note that this is still ecm transactions from the user point of view, so time stamps and
517 durations reflect when the client sessions sent ecms to the proxy, not when the proxy sent it on to the connector.
518 In the status web, a link to this log appears under each connector in the status view.
519
520
5210.8.4 - 2008-05-27
522
523- Fixed: Old ecm replies being processed as card data on rapid newcamd reconnects (caused parse errors and bogus ca-id
524 for connectors, leading to them being disabled).
525- Fixed: Newcamd clients sending multiple async requests to the proxy would get incorrect/duplicate sequence numbers
526 in their replies (effectively making the proxy incompatible with async mode).
527- Fixed: Events for connectors with no profile configured (i.e those with auto-detect) were hidden even from admin in
528 the web error-log.
529- Fixed: Connection failures occuring during startup are no longer filtered in the web view. Also replaced the firefox
530 alert error for when the web script can't reach the proxy to something less cryptic.
531- Connector re-connect behaviour on login failure improved (no longer logs disconnects which would cause 3 sec loop).
532- Flag T is now only used for actual timeouts when forwarding, transactions affected by aborts/disconnects will receive
533 flag A instead.
534- Service names will now include a [R] prefix for radio and [HD] for hdtv (based on the type from the services file).
535- Added more details to the LoggingFilter output (sequence numbers, sids, sessions) to improve client troubleshooting.
536- Duplicate newcamd messages (with the same sequence number) are now logged as warnings for the CWS communication.
537 Also, if debug logging is set the same is done for client communication (although these can occur normally when
538 zapping or due to local network lag/congestion and don't necessarily mean trouble).
539 NOTE: If a server returns the same sequence id twice (without having been sent that) it indicates something went
540 wrong on the server side, either some kind of overload situation or outright bugs.
541 If it repeats it needs to be investigated further and resolved.
542 The proxy will now also check to make sure the sid in the reply matches the one in the request, to help identify
543 error replies.
544- For case-insensetive user managers (e.g SimpleUserManager), the stored case of the user name is now used instead of
545 the one supplied by the client for the login.
546
547- Changes to proxy.xml:
548 Added: Element <log-sid-mismatch> to <connection-manager> (true/false, default: true). Allows turning off logging
549 warnings when sid in the server reply doesn't match what was in the request.
550
551
5520.8.3 - 2008-04-15
553
554- Fixed: CtrlCommands caused NPE if status-web was disabled or had failed to start.
555- Fixed: Logging in case newcamd card data could not be parsed (WARNING level + the offending data now logged).
556- Fixed: Services from different profiles could be merged in the watched-services xml reply if they had the same name.
557- Fixed: Minor pending ecm leak in ClusteredCache.
558- Changed asynchronous newcamd mode to be off by default as it caused problems with some servers (you now have to
559 explicitly set <asynchronous>true</asynchronous> for each connector if you want to use it).
560- Further tweaked utilization estimates, now using different methods depending on async/sync mode.
561- Made it possible to manually override the service maps for each connector, by specifying sid lists. Useful for
562 situations where the automatic service discovery is unreliable. Services that aren't manually specified will still
563 be probed for automatically.
564- Status web now shows services with full information (sid and profile), space and context permitting.
565 Additionally, services listed per connector will be highlighted blue if a forward occured for that service in the
566 last max-cw-age seconds (allows you to see roughly which services the ecm-load value refers to).
567
568- Added a mechanism to better handle overload situations (= only congested/timeout state connectors are available).
569 If this is enabled (which is default), the proxy will avoid forwarding until the situation resolves itself,
570 by returning empty cannot-decode replies to clients (flag N). Note that this typically causes clients to retry
571 repeatedly (e.g with a 1 sec interval), but that these retries will also fail immediately until there is card
572 capacity available.
573 This may help stabilize traffic in overloaded shares (or shares where cards are temporarily lost).
574 Clients that don't retry the same ecm immediately when receiving an empty reply will likely freeze.
575
576- Changes to proxy.xml:
577 Added: Element <hard-congestion-limit> (true/false, default: true) to <connection-manager>. See above.
578 Changed: Element <asynchronous> for <newcamd-connector> default value changed to false.
579 Added: Elements <can-decode-services> and <cannot-decode-services> to all connector types. Optional lists of
580 sids (hex) allowing manual overriding of the automatic service mapping. Note that services already known to decode
581 on a connector will not be affected by the cannot-decode-services list (until a manual reset is performed, or the
582 corresponding .dat file is deleted from the cache dir).
583
584- Changes to the http/xml api:
585 Added: Attribute 'hit' to <service> elements for <connector> (cws-connectors reply). If this attribute is present
586 and the value is 'true', it indicates there was a forward for this service within the last max-cw-age seconds.
587
588
5890.8.2 - 2008-03-07
590
591- Fixed: ArrayIndexOutOfBounds on emm forwards to newcamd connectors set to asynchronous false.
592- Fixed: Connectors getting stuck in unresponsive state for extended periods (keep-alives are now sent as before 0.8.0).
593- Fixed: CWS average processing time included send-queue time (even with async false), restored old behavior.
594 The utilization and capacity calculations have also been tweaked to use the most optimistic estimates (since averages
595 will vary in async mode), this may help with async connectors showing more utilization than they actually have.
596
597- Congestion warnings are now logged only when no alternative connectors exist. Also, the utilization estimate is now a
598 factor in determining congestion (i.e > 100% over the last 60 secs = congested even when there are 0 pending requests).
599- Made it possible to configure the threshold for logging CWS timeout events (default 1, was 2 before 0.8.0).
600 The number of timeouts to allow in a row before disconnecting can also be set (default 2, was 3 before 0.8.0).
601- The max-connections value (per user) now defaults to the number of profiles the user has access to, or the total
602 number of active profiles if there are no restrictions. Previous default was 1, if you want to keep that limit you
603 will now have to explicitly set every user to max-connections="1" in the user manager.
604- Flag T now means only this: Timeout when forwarding (no response from CWS within time limit, i.e max-cw-wait).
605- New flag S introduced: Timeout in send queue (when trying to forward to connector, should normally not occur).
606- The log event (level FINE) that occurs when client sessions end now contains a summary of the session state, to help
607 show why the client may have disconnected. If the user has debug="true" this is logged with level INFO.
608- If the ca-id for card data (received during login for a newcamd connector) is 0000, the proxy will fail the login and
609 try again later instead of disabling the connector. This may help with re-init card issues that cause servers to
610 temporarily return empty card data.
611 If override-checks is true then this check is also skipped and the 0000 data accepted as valid.
612
613- Changes to proxy.xml (optional additions only):
614 Added: Element <event-threshold> to <logging>, defining how many failures are required to create a CWS event.
615 Attributes are: min-count (default 1).
616 Added: Element <timeout-disconnect-threshold> to <connector-manager>. The number of timeouts before a connector
617 is closed and reconnected (default 2).
618
619- Changes to the http/xml api:
620 Added: Attribute 'ecm-load' to <connector>. This shows the ecm count over the last max-cw-wait seconds. This provides
621 an absolute measure of the load on the connector (whereas the utilization is relative to the estimated capacity).
622
623
6240.8.1 - 2008-02-28
625
626- Fixed: Potential deadlocks in relation to network timeouts, introduced in 0.8.0 (sessions waiting on connectors
627 waiting on sessions). Caused eventually fatal thread leaks.
628- Fixed: Javascript alert box infinite repeat for some status web errors that resulted in logout.
629- Fixed: The improved probing could still generate duplicates.
630- Modified to work with JamVM (1.4.3+)! If you get "Unrecognized option" socket exceptions for the connectors, add
631 the following attribute to the connector definitions (both radegast and newcamd): qos-class="none"
632 For more information on JamVM and gnu classpath see:
633 http://jamvm.sourceforge.net/
634 http://www.gnu.org/software/classpath/classpath.html
635
636- Added a quick-start option: if the proxy is started with no proxy.xml config file, one will be generated.
637 The generated config will use recommended defaults, and a single profile with two connectors (disabled).
638 Most non-essential elements can now be omitted from the config, defaults will be used if they are.
639 Some defaults have been altered (e.g retry-lost-services now defaults to true, missing services files are ignored etc).
640
641
6420.8.0 - 2008-02-25
643
644- Switched to fully asynchronous newcamd communication with servers. This should significantly increase throughput, but
645 may not work with all servers. If it fails or behaves erratically with your server of choice, the old behaviour can
646 still be used if the newcamd-connector element contains <asynchronous>false</asynchronous>.
647 Radegast connectors always use the old synchronous mode.
648
649- More changes to support other jvms (removed references to sun base64, and httpd now tries multiple ssl providers).
650- Added a javascript xml-editor to the status-web, for quick config updates (based on codemirror, source included).
651- Forced parsing of proxy.xml to always use UTF-8 regardless of system locale (when installed from both file and web).
652- Cleaned up web backend and made it possible to extend it from user code (see ClusteredCache, XmlUserManager source).
653- Cleaned up the client side scripting for the status-web, it now uses xslt to generate the markup (see xslt dir in war).
654- All user-log transactions will now contain cws-name (if it was a transaction that involved a forward).
655- Added a new user-warning-log with potential problems from all user recent transactions (40 most recent).
656- Transactions marked as warnings will now contain additional debug information, and show the time spent on each stage:
657 in cache, cws send queue, cws reply wait, client write back.
658- Tweaked probing to avoid multiple probes for the same service and connector, and avoid problems under high load.
659- Fixed: Now possible to add new ca-profiles without restarting.
660- Fixed: Transaction tracking now correctly deals with overlapping/asynchronous newcamd traffic. Flag E is now always
661 included if the client reply was empty, regardless of the cause.
662- ClusteredCache will now indicate in the log which remote proxy has the wrong version (SEVERE event on startup).
663 Additionally the cache stats on the web will contain a version-mismatch property with the same IP.
664- Status web section "Sessions" (previously Users) will now contain last-seen information for disconnected users and
665 an option to show the idle sessions. Note: No last-seen data will be shown if all known users are connected.
666 This section is now shown for regular users (non-admin) but will only list information related to their own sessions.
667
668- Made it possible to read connector definitions from an external or remotely hosted file, similar to XmlUserManager.
669 NOTE: This doesn't mean you should have multiple proxies reading from the same connector file, use one for each.
670 As with the XmlUserManager and the ClusteredCache tracker file, the blowfish option is there strictly to allow the
671 files to be hosted in a public place (it adds _zero_ security if the files are hosted with the proxy).
672 Using an external connector config also makes it possible to keep connector definitions in a database, just provide
673 a php/jsp/asp page to render the xml on demand.
674
675- Changes to proxy.xml:
676 Added: Element <warning-threshold> to <logging>. Defines which transactions should be considered potential problems.
677 Attributes are: bad-flags (string list of all that should qualify), max-delay (in ms).
678 This setting determines what will show in the user-warning-log for profiles that have debug="true".
679 Note: changing it will not affect already recorded events.
680 Added: Element <external-connector-config> to <connector-manager>. Specifies an external source for connector
681 definitions with the following elements: connector-file-url, connector-file-key, update-interval (minutes).
682 Added: Element <asynchronous> to <newcamd-connector> (true/false, default: true). A way to disable asynchronous mode.
683
684- Changes to the http/xml api:
685 Added: Example test page: /api-test.html
686 Added: New status command: fetch-cfg (no params). Returns the currently used config file as is (no cws-status-resp).
687 Added: New http end point /cfgHandler (for posting updated config xml, as an admin user with http basic auth login).
688 Added: New status command: user-warning-log. This log is aggregated and contains the 40 last potential problems from
689 the user transaction logs. Only profiles with debug="true" will trace these events. Events of the same type from
690 the same user are merged and tagged with a count (time stamp and transaction shown are always from the last such
691 event logged). Any transaction that fits the warning-threshold critera will qualify (see config). Only admins will
692 see warnings from other users traffic.
693 Removed: Attribute 'admin' from <proxy-users> reply when requesting multiple or all users (i.e. no name specified).
694 Caused problems for some user managers and wasn't used for anything.
695 Added: New status command: last-seen (params: name). Returns information about currently disconnected users.
696 This data is tracked by the session manager and saved regularly as etc/seen.dat. Delete it before startup to clear
697 the seen history.
698
6990.7.6 - 2008-02-02
700
701- Changed the example LoggingFilter into something actually usable. It now logs just the raw messages (minus encryption)
702 from user sesssions (as RECV) and their responses from the proxy (as SENT).
703- Removed all unnecessary java.util.logging experiments, to be compatible with gcj/gij and possibly other jvms.
704- Fixed: Removing au-users no longer requires restart.
705- Fixed: Adding/removing/changing listen ports should now work without restart. Disabling a profile will close the
706 ports (but existing sessions are not affected until kicked). Any change to a ca-profile config will result in the
707 listen ports for that profile being closed and reopened.
708- Fixed: Status-web httpd no longer logs to sysout if its log file is disabled (removed from config).
709- Fixed: Status-web jscript continues to request xml after connection errors (now logs out instead).
710- Fixed: Cache timestamp bug that could sporadically delete all cached ecms except one.
711- Fixed: ECM interval for radegast sessions (0 was always shown).
712- Fixed: Mgcamd OSD message sending would fail if radegast sessions were active.
713- Exposed the automatic rotation features of the logging api, in case anyone is serious about using the logs for stats.
714 A maximum size for all file logs as well as a number of files to cycle through can be specified (see below).
715 Additionally, any active file logs are now re-initialized when the config is updated/touched.
716- Added JVM stats to the status-web title for all views (os, version, heap used/allocated, thread count etc).
717- Enhanced the embedded httpd (keep-alive connections, gzip content-encoding).
718
719- Changes to proxy.xml:
720 Changed: Renamed <exclude-services> setting to <reset-services>, to reflect what it actually does.
721 Services in this list will also no longer cause "lost service" warnings.
722 Changed: <auto-exclude-threshold> is now <auto-reset-threshold>.
723 Added: Attribute 'bind-ip' to listen-ports. Optional local ip to bind listeners to (default is all, i.e 0.0.0.0).
724 Added: Element <bind-ip> to <status-web>. Optional local ip to bind httpd listen port to.
725 Added: Element <bind-ip> to <rmi>. Optional local ip to bind all rmi-related listen ports to.
726 Added: Attributes 'rotate-count' and 'rotate-max-size' (in kb) to <log-file> (for both main log and status-web).
727 E.g: setting count to 3 and max-size to 2048 will cycle between file.log.0, file.log.1 and file.log.2 when they
728 reach 2 megs. The file currently in use will be indicated by a separate .lck file.
729 Restart is required to change the log rotation.
730
731- Changes to the http/xml api:
732 Added: Element <jvm> to <proxy-status>, with attributes: name, version, heap-total, heap-free, threads (count), os
733 Added: Attribute 'version' to <proxy-status> (csp version)
734
735
7360.7.5 - 2007-10-17
737
738- Added another example user-manager implementation: com.bowman.cardserv.XmlUserManager (see separate README).
739- Fixed a bug that caused send attempts for queued ecms/keep-alives on unitialized NewcamdConnections (NPE in sysout).
740- Fetching the service list for a connector that was in a reconnect loop (via http) would cause NPE, fixed.
741- Added support for receiving sid in radegast messages (as sent by mgcamd 1.25+ and maybe others, in field 0x21).
742 In the unlikely event that there are radegast clients out there that will put something other than the sid in this
743 field, sid parsing can be disabled by setting the new attribute sid-in-0x21="false" for the radegast listen port.
744- Added support for sending sid in radegast messages, using the same method as above.
745- Fixed a bug that prevented kicking of idle user sessions.
746- Fixed stats based on sliding window averages (utilization, intervals) to properly show ceased activity when
747 appropriate. Utilization now actually shows the current load (over the last minute), and the average utilization
748 since connect only takes successful transactions into account (as failed tend to be nearly instant).
749- Added a capacity estimates to the status web (for total, cws and profile). This is simply the following calculation
750 (since people seem to have a hard time grasping this basic fact): max-cw-wait / processing time
751 E.g: 10s / 900ms ~= 11 transactions per CW validity period
752 (= 11 simultaneous clients in a worst case scenario with no cache hits, or 11 different services processed).
753 When total capacity is greater than the number of services for the provider (or providers if using multiple profiles),
754 you can have an infinite number of clients. Total estimates will only make sense once all cards have handled at least
755 one ecm transaction.
756- Added red hilighting to potential problem values in the user list of the status web, along with capacity estimates
757 and service-mapping count (per profile). :)
758- Made it possible to control which card-data is returned to clients, as a setting per newcamd listen-port. Previously
759 the proxy would use data from one of the cards in the profile, more or less at random. Card-data can be specified by
760 connector name (to get the data from) or by a file name. The proxy will dump card-data files when connecting to
761 cards (in the etc dir). This feature is only useful for ca systems where the providers/idents on the cards matter
762 to the client, and affects only the newcamd protocol (see protocol.txt for card-data format).
763- Au-users are now given card-data only from the card they're meant to update (if several, they receive the first
764 available). If cards were not being updated properly despite receiving emm's, this was a possible cause. Can be
765 overridden by the above card-data feature if the attribute override-au is set to true.
766- Added validation checks that ensure all cards in a profile are identical. Unless the attribute override-checks is
767 set to true for the connector, a card will be _disabled_ when it has a differing ca-id or a different provider ident
768 list than another already connected card in the profile. Warnings will be logged. Under normal circumstances with
769 most ca-systems, differing cards should require separate profiles.
770- "Negative" forward notifications are now sent by the clustered cache (i.e one proxy will notify another that it
771 couldn't provide the reply that it previously indicated it was going to handle). Thus giving the remote proxies
772 a chance to get it elsewhere.
773
774- Changes to proxy.xml (only optional additions):
775 Added: <card-data type="connector|file|empty" name="connectorname|filename"/> for <newcamd> listen ports. Only use
776 this if you need to make sure that clients connecting to a specific newcamd port always get the same exact card-data.
777 Added: <hide-unknown-services> to section mapper (true/false, default: false). Hides services if no name for them was
778 found in the services file (only affects remote monitoring, e.g xml commands like all-services and cws-connectors).
779 Added: <block-services> to section mapper. List of sids (hex) that the proxy will always immediately return empty
780 results for, and not probe cards/attempts forwards. Can be used to optimize handling of services known not to
781 exist on any card and reduce probing. E.g: you know there are cards available to handle all services except 3, then
782 add those to the block list and the mapper will stop trying.
783 Added: Attribute 'override-checks' for <newcamd-connector> (true/false, default: false). Set to true to skip card
784 validation checks. If you use this for one card in a profile you should probably enable it for all of them.
785 Added: <congestion-limit> to section connection-manager. If you use different metric priorities for CWS connectors,
786 then this allows you to set what the maximum queue-time estimate can be before a connector is considered
787 congested (and higher metric connectors are used instead). This can lower response times but should be used with
788 care. Value (in seconds) must be between max-cw-wait/2 and max-cw-wait. Reasonable values could be 4-8.
789
790- Changes to the http/xml api:
791 Added: Attribute 'capacity' for <proxy-status>, <profile> and <connector>.
792 Added: Attribute 'active-sessions' for <proxy-status>. Count for sessions currently generating traffic (not idle).
793 Added: Attribute 'mapped-services' for <profile>. The number of services known by the service-mapper for this profile.
794
795
7960.7.4 - 2007-09-04
797
798- Automatic profile assignment based on ca-id is now done for each cws reconnect, not just the first one.
799- No longer ignoring HD services when parsing (dvb service types 0x11 and 0x17, for mpeg2 and "advanced codec" hdtv).
800- "Successful" cannot decode replies are no longer counted as ecm failures in the stats.
801- Additionally tweaked load-sharing (0.7.3 may have had a serious bug that only manifested under heavy load on 3+ cards).
802- Emm's are now properly acknowledged to the client even when sent by non au-users (without forward of course).
803- Fixed several potential problems with radegast sessions (clients using radegast towards the proxy).
804- Added a delay on startup, listen ports will now not be opened until the cws connector manager has had a chance to run
805 through one connection attempt for each active cws connector (this will prevent the service mapper from removing
806 entries for "unknown" connectors that simply haven't been connected for the first time yet).
807- Added a real time negotiation procedure in the clustered cache to maximize use of cache-sharing. Using this adds a
808 fixed delay (e.g 100 ms) to _every_ transaction. The time is used to collect cache notifications for a pending ecm
809 from all proxies in the cluster, and ensure that only one of them proceeds with querying a card. Highly experimental,
810 not quite sure yet whether this is useful in a real life scenario. The multiple proxies will attempt to find which
811 one is best suited to handle a given request (based on estimated queue time and whether the services is known to
812 exist on a local card or not). If in doubt, leave it alone (it's enabled with <sync-period> for the clustered cache).
813- Added a preconfigured java-service-wrapper setup for running the proxy as a service on w2k/2k3/xp/vista. See README.
814- The 'last-transaction' time per session now includes any time it took to send the reply back to the client. This
815 means it is no longer depending exclusively on the proxy response time, but also on the client connection.
816- Changed CWS average processing time to current processing time (thats what it was before as well, just a misleading
817 label). Average processing time is now the true total average for all ecm's processed since connect (and utilization
818 will now show both versions).
819- Made it possible to use any number of listen ports for each profile, each can have their own protocol and allow/deny
820 lists as well as their own des-key/noencryption settings (and any other protocol-specific data). Adding/changing ports
821 while running should work now too.
822- Fixed weird synchronization issues with the session manager (caused NPE in various places and blocked logins,
823 especially in conjuction with heavy use of the status web).
824
825- Changes to proxy.xml: (mostly optional additions)
826 Added: Attribute 'debug' for <user> (SimpleUserManager), set to true to enable log-ecm, log-emm and log-zapping but
827 only for this user. NOTE: For protocols like radegast that have no user concept you can still use this and other
828 special attributes, by defining a dummy user with a name like: protocol@source.ip.address (e.g radegast@192.168.0.3)
829 and a dummy password. It won't be used for login auth but attributes like map-exclude and debug will be applied.
830 Added: <ip-filter> to <filter-config> for the LoggingFilter, if specified then only traffic from the matching ip will
831 be logged (? * wildcards supported, e.g: 192.168.1.*).
832 Added: <cannot-decode-wait> to <connection-manager>. This adds a configurable delay when service mapping determines
833 that there are no cards available that can decode a given ecm (or there are no cards at all). Instead of immediately
834 responding with a cannot-decode reply, the proxy will wait the specified number of seconds and then check the cache
835 again. This increases the chances of a cache hit through sharing. It will have no effect on cache-only profiles
836 since these always wait as long as possible, but if you're using cache sharing in combination with local cards you
837 should try this and set it to 1-4 seconds.
838 Added: <sync-period> to <cache-config> (ClusteredCache).Set to larger than 0 to enable the experimental synchronized
839 cluster arbitration procedure. Value in milliseconds (try somewhere around twice the round trip ping between the two
840 furthest/slowest proxies in the cluster).
841 Moved: Attribute 'no-encryption' for <newcamd> is no longer an attribute, it should now be an element within newcamd,
842 i.e: <newcamd listen-port="1234"><no-encryption>true</no-encryption></newcamd>. It is now possible to have
843 multiple newcamd ports for a single profile, and set no-encryption per port.
844 Added: Attribute 'debug' for <profile>, set this to true to enable storing the last 100 transactions for each user
845 session (in order to use the user-log http/xml command for troubleshooting).
846 Added: <retry-lost-services> to section mapper (true/false, default: false). Whenever the service-mapper registers a
847 service lost from a card that could previously decode it, it will register a background probe to see if it returns.
848 The status for the service on the particular card in question will be reset with an increasing interval (doubles
849 every time, starting at 5 minutes after it was lost and ending if it hasn't been found after 48 hours).
850 NOTE: This only makes sense if there are multiple cards in the profile, otherwise lost services would be found
851 within minutes when someone tried to watch them, through the auto-reset-threshold.
852
853- Changes to the http/xml api:
854 Added: Attribute 'flags' for <session> (proxy-users). This will contain information about the last ecm transaction
855 in the form of one or more of the following chars:
856 C = Cache hit (local), R = Cache hit (received from remote cache), F = Forward occured, Y = Forward retry,
857 N = Cannot decode (mapping says service not on any card), T = Timeout when forwarding, O = Timeout in cache,
858 G = Congestion when forwarding (time > max-cw-wait/2), I = Instant cache hit (no waiting at all in cache),
859 W = Triggered cannot-decode-wait, X = Cache hit after failed forward, E = forward returned empty (cannot-decode),
860 Z = SID changed (compared to previous transaction), P = Triggered probing of one or more cards,
861 D = The user session disconnected before it could receive the reply (likely reached the client ecm timeout),
862 + = Caused an addition to the service map (found channel), - = Caused a removal from the service map.
863 Examples (flags in the attribute string are not shown in chronological order):
864 +FPZ (client changed to a service that had uknown status on some cards, triggering probes, one of which
865 found the service where it wasn't previously known to exist).
866 CI (local instant cache hit, both ecm and cw were immediately available in the cache when the client asked)
867 RZ (cache hit on changing service, and the cw reply was provided by a remote proxy)
868 FO (client was held in cache waiting for a reply that never came, and eventually fell back to forwarding)
869 The statusweb user section has been updated to show these flags, and a info level log printout has been added (only
870 when log-ecm is true).
871 Added: New command 'user-log'. This will show the status of the last 100 ecm transactions completed on any of the
872 selected users sessions. Admins can specify user name with the 'name' parameter, others always get their own log.
873 This allows you to troubleshoot a specific glitch experienced by one user, find the corresponding ecm transaction
874 and see exactly what caused it to fail (at least if you check within 100*10 seconds, i.e about 16 mins).
875 The per-user storage will only be updated if the profile has debug="true" set. A form for this command has been
876 added to the admin page of the example-web.
877 Changed: <profile> now has a single 'listen-ports' attribute that contains a string list that will indicate protocol
878 and port number for each of the defined ports, instead of the previous radegast-port/newcamd-port attributes.
879
880
8810.7.3 - 2007-07-28
882
883- Fixed a bug that caused web logging to switch to sysout after changing the httpd port number.
884- Fixed problem with rapid cws reconnect that caused an IndexOutOfBoundsException if there was pending traffic.
885- Fixed NPE on NewcamdCws connect if the socket was unexpectedly closed during the login procedure.
886- Fixed last-transaction time for a user session getting confused by other message types (non-ECM).
887- Improved load balancing: queue size has been redefined to take the average processing time into account, meaning that
888 in theory a queue size of 3 on a fast card can now be considered faster than a queue size of 1 on a slow card.
889
890- Updated the example web page to include several user contributions, including a user/session section and re-use of
891 the xml parser object (to stop IE from leaking memory with each xml pull). I'm keeping my logo though. :)
892- Added a filename translation servlet for picon images. See /picon/readme.picon.txt in the war for details.
893
894
8950.7.2 - 2007-06-10
896
897- Fixed a bug that would cause emm's to be forwarded to connectors that weren't connected (caused NPE stacktrace in log).
898- Fixed NPE in radegast response parsing when no matching pending request was found.
899- Fixed a serious cache sharing bug that could prevent remote cache data from being used between servers with different
900 system locales set for the jvm (language/regional settings).
901- Fixed another serious cache sharing bug that meant system clocks had to be synchronized to within the max-cw-age time
902 across all servers, or cached cw's could be deleted before they were used.
903
904- Added a cache-only (card-less) mode. A profile that makes use of this will accept connections even if there are no
905 cws connectors ready (or even defined). It will rely entirely on cache sharing to handle requests.
906 This allows a frontend proxy to be set up where untrusted users could get access to anything cached, without being
907 able to affect the traffic load on any cards. Such a card-less frontend proxy would only receive cache data (one way,
908 use the receive-only ClusteredCache setup by leaving out remote-host and remote-port) from one or more other proxies,
909 thus completely isolating clients from the real proxies and their clients/servers.
910 As long as the backend proxies have enough users to statistically ensure that all the profiles services are cached
911 at any given moment, all services would also work for clients in the frontend proxy.
912 NOTE: When there are no newcamd connectors in a profile, clients will receive dummy card data on connect (empty).
913 This may not be good enough for all clients/ca-systems, but it works for me. If you do have connectors defined
914 and cache-only mode set, the connectors will not be used for traffic (only for card-data).
915
916- Changes to the http/xml api:
917 Added: Attribute 'profile' to <service> for the commands watched-services and all-services, since these lists can
918 contain services from multiple profiles if the calling user has such access.
919 Added: Attribute 'cache-only' for <profile> (ca-profiles command).
920
921- Changes to proxy.xml:
922 Added: Attribute 'log-zapping' for <logging>. Set to false to disable the log entries for when users switch service.
923 Added: Attribute 'cache-only' for <profile>. Set to true and the profile will accept connections even without
924 card connectors. This will also prevent the "no available card" warnings, the proxy will instead silently return
925 cannot decode for each cache miss.
926 Added: Attribute 'no-encryption' for <newcamd> (in profile). Setting this to true means the proxy will handle
927 unencrypted newcamd traffic on this listen port. NOTE: no clients support this, its just for debugging.
928
9290.7.1 - 2007-05-19
930
931- Major structural changes to handle multiple protocols (besides newcamd), expect bugs.
932- Fixed a bug where connectors could be flagged as congested and never recover.
933- ECM cache hits (INFO level printout) now mentions if the hit was received from a remote cache (and which one).
934- Added basic radegast support for both incoming and outgoing connections. Since this protocol doesn't have user auth
935 or includes SID, clients should avoid using it if they can. Using cws connectors with the radegast protocol
936 should not have any obvious drawbacks however. A consequence of this is that it becomes possible to use a "radegast
937 only" client towards cardservers that only allow the newcamd protocol (and vice versa) using the proxy to convert.
938 If a radegast listen port is configured, the accept/deny list is the only access control since there is no user auth.
939 NOTE: If using only radegast cws connectors, and connecting to the proxy with a newcamd client, the proxy will
940 return dummy card data to that client (a single provider with ident 0, but including the ca-id for the profile so this
941 should probably be set correctly).
942
943- Added an option for completely disabling the service-mapping. Doing so means the proxy will no longer attempt to find
944 out which services exist on each card. Instead it will assume all cards in the profile are identical and only apply
945 load balancing. This way effective clusters can be achieved even with protocols or clients that don't include SID,
946 providing the cards all have the same services. It can also be useful for troubleshooting in small clusters.
947 Profiles for which the mapping has been switched off will not show any service lists in the http/xml api or webgui.
948
949- Changes to the http/xml api:
950 Fixed: Duration time strings (longer than 1 month was previously broken?).
951 Changed: Any user@host:port information for cws connectors from the error log is now only shown to admin users.
952 Added: Utilization percentage per cws connector, based on the current average ecm time. This shows the % of the
953 total time since connect that the card has been spending on actual processing (rough estimate).
954 Changed: getUserCount() in the remote api renamed to getSessionCount() since thats what it actually is.
955 The <users> element in the proxy-status reply has been changed to <sessions> to reflect this.
956 Added: Attribute protocol added to <session> and <connector> (a string with either Newcamd or Radegast for now).
957 The example web page has updated to reflect this in the status CWS list.
958
959- Changes to proxy.xml: (old configs WILL need to be edited)
960 Added: Separate listen ports for <newcamd> and <radegast> protocols, per profile. Each with an optional accept/deny
961 list that uses ip masks with wildcard support (separated by space). See the example configs for details.
962 Changed: Moved <des-key> from <profile> to <newcamd>.
963 Removed: <ban-list> from <profile>, replaced with the allow-list/deny-list elements of the new newcamd and
964 radegast elements, e.g: <newcamd listen-port="1234"><deny-list>10.0.0.*</deny-list></newcamd>
965 Changed: <connector> elements in <cws-connector> replaced with either <newcamd-connector> (as before) or
966 <radegast-connector> (profile attribute mandatory, and only host/port).
967 Added: Attribute 'enabled' for <mapper> in <service-map>, to turn of mapping entirely for one or all profiles.
968 Added: Attribute 'hide-ip-addresses' for <logging>, this will replace all IPs in logs like so: xxx.xxx.xxx.123
969 This doesn't apply to the web-access.log, switch that off by removing the rmi/log-file element when using this
970 option. Debug output or log levels lower than INFO are also not affected.
971 Changed: Attribute ca-id for <profile> is now mandatory (it simplified the radegast implementation).
972
9730.6.3 - 2007-03-03
974
975- Changed load balancing to consider queue size before other factors.
976- Fixed potential false "connection from different IP" errors for rapid client reconnects.
977- Fixed handling of missing SID (service maps ignored but load balancing applied). Clients that don't send SID should
978 work fine now providing all the cards in the profile have the same services (or there is just one card).
979 Mechanisms to more effectively deal with setups where this is not the case will be added in a future version.
980- Enigma service files are now monitored for changes and re-read if changed. This allows for automated updates of the
981 files, i.e by cron'ed ftp/scp. They are of course also re-read when proxy.xml changes, as before.
982- Added INFO level logging for web logins (see the web-access log for more details) and WARNING for failed attempts.
983- Changes to the http/xml api:
984 Added: Simple admin section in the example web page, for executing the control commands.
985 Changed: Services with no name/type information are now included (as "Unknown (id)") in xml replies.
986 Changed: Cleaned up most of the javascript for the example web page and fixed some minor firefox issues.
987
988- Changes to proxy.xml:
989 Changed: Attribute 'provider' for <services-file> can now contain a list of names (separated by space). Use this
990 in case services in the file have variations in the provider name (or lists some as unknown even though
991 they should be part of the same provider subscription).
992 Added: <unknown-sid> to section mapper. Defines a special SID that will be sent to servers when SID is 0 (unknown).
993 Client requests for this SID will also be treated as if it was 0. This can be used as a workaround for
994 servers that require a non-zero SID, and for clients that send a fixed special SID instead of 0 when service
995 is unknown (e.g cardlink). Make sure that this isn't set to a real SID that exists in the services file.
996 Can be specified globally or per profile like all other mapper elements.
997
998- Misc minor fixes and improvements.
999
10000.6.2 - 2007-02-10
1001
1002- Fixed rare ConcurrentModificationException related to the average calculations.
1003- Changed the timeout for keep-alive replies to 3 seconds instead of using max cw wait.
1004- Fixed false timeouts that could theoretically occur in the connectors even when max cw wait wasn't exceeded.
1005- Improved the default INFO level logging to make it easier to spot problems.
1006- Changes to proxy.xml:
1007 Added: Attributes 'log-ecm' and 'log-emm' to section logging (true/false, default: true).
1008 Added: <log-missing-sid> to section mapper (true/false, default: true). Allows hiding the [0] SID warning.
1009- Changes to the http/xml api:
1010 Added: 'last-transaction' field to proxy-users. Shows how long the last perceived ecm -> cw roundtrip time was (ms).
1011
10120.6.1 - 2007-02-05
1013
1014- Added a getUsageStats() method to the CacheHandler interface. Caches can return whatever relevant information they have.
1015- Improved the queue handling for outbound ecms. Queue size and average response time is now used for load balancing.
1016- Changed timeout handling for cws connectors. If a timeout occurs the connector will be removed from load balancing
1017 until it is responding again (unless it is the only available connector for a given request). Keep-alives will be sent
1018 until it either responds or exceeds the maximum number of timeouts and is disconnected. This should help minimize the
1019 impact of shaky connections in setups where there are multiple connectors of the same type available.
1020- Remote api extended with kickUser and shutdown methods.
1021- Changed average ecm interval and average processing speed to count for the last minute rather than total since connect.
1022 This also applies to the rate limit feature in the UserManager interface.
1023- Changes to proxy.xml:
1024 Removed: <peer-proxy-url> from status-web section. This no longer made much sense.
1025 Added: Attribute 'map-exclude' for the auth-config section (SimpleUserManager). Stops a user from causing changes
1026 to the service maps. Can be useful in large clusters for clients that misbehave and send a lot of bad ecms.
1027 Changed: <tracker-update> for ClusteredCache can now be set to 0 to disable auto updates (i.e only update when
1028 proxy.xml is updated).
1029
1030- Changes to the http/xml api:
1031 Added: New command api, accessible by admin users only. 4 commands so far: reset, kick, shutdown and osd-message.
1032 - Reset will clear the service map for a specific service on all cards, or for all services one card.
1033 Params: name (cws connector name) or profile + id (service id in hex or decimal).
1034 GET example: /xmlHandler?command=reset&profile=myprofile&id=0x04F3
1035 POST example: <cws-command-req><command command="reset" profile="myprofile" id="1267"/></cws-command-req>
1036 - Kick will close all sessions for a specific user (mainly a debugging tool, the user cam will just reconnect).
1037 - Shutdown will stop the proxy node.
1038 - Osd-message sends a Mgcamd osd message to any matching active user sessions (with client id = Mgcamd).
1039 GET example: /xmlHandler?command=osd-message&name=username&text=hello%20there
1040 Name can be omitted to send to all users. This is experimental, use with care.
1041 Note: the command api responses will contain the element <cws-command-resp> instead of <cws-status-resp>.
1042 Added: New status command 'cache-status'. Will show usage counters for the currently used cache implementation.
1043 Changed: Timestamps now use RFC822 format, and any previously included duration has been moved to a separate field.
1044 The timestamp fields have also been renamed to show what they represent (i.e started or connected).
1045 Changed: Command 'tv-services' changed to 'all-services'.
1046 Added: New status command 'watched-services'. Returns currently watched services with a user count. :)
1047
1048- Added a view of the 'watched-services' to the channels section of the example web page.
1049- Added cache-status to the status section of the example web page.
1050- Added some reasonable usage stat counters to DefaultCache and ClusteredCache.
1051- Added isMapExcluded() to UserManager interface, return true to stop a user from discovering new services or changing
1052 the status of existing ones (map failure counters etc will not be updated as a result of ecms from this user).
1053- Improved some of the INFO level logging to more clearly show the new load balancing and timeout handling in action.
1054
10550.5.1 - First public release
1056
1057- Changes to settings for proxy.xml (see config/proxy-reference.html for full details):
1058 Removed: <default-profile> from rmi section. Remote api now allows specifying a list of profiles per method.
1059 The http/xml api and web page will use this to only show info for profiles accessible by the current user.
1060 Added: <allowed-ip-masks> to rmi section. List of ip masks separated by space that should be allowed to use the
1061 remote api.
1062 Added: <default-client-id> to section connection-manager. Allows setting the 2 bytes that is used by newcs and
1063 others to identify the type of client. Beware, this only works as long as the server doesn't modify its
1064 behavior or makes use of client-specific features based on this value (for example identifying as mgcamd
1065 will cause newcs to do this and no longer be compatible with the proxy).
1066 Added: <client-id> to section connector. Same as above but overriding the default for one connector.
1067 Added: <au-users> to section connector. List of users allowed to send AU to this connector. All EMM's clients from
1068 these users send to the corresponding profile will be forwarded to this connector. The old behaviour with
1069 matching connector and user name for AU has been removed.
1070 Added: Attribute 'metric' for <connector>. Allows grouping connectors together for preference in the load-balancer.
1071 See proxy-reference.html for examples.
1072 Added: Attribute 'admin' for the auth-config section (SimpleUserManager). Set to true to mark a user as
1073 administrator. Only used by the http/xml interface so far.
1074 Added: <ssl> to section status-web. Allows HTTPS instead of HTTP for the http/xml api and web page. This requires
1075 a java keystore file with a certificate to be specified as well, e.g:
1076 <ssl enabled="true"><keystore password="123456">etc/keystorefile</keystore></ssl>
1077
1078- Changes to http/xml api:
1079 Added: Only info from profiles that the current user has access to are shown. There should be no way to tell other
1080 profiles even exist in the config.
1081 Added: New HTTP GET based version of the xml api. Access /xmlHandler?command=command&paramName=paramValue to get
1082 the same reply that posting <command include="true" paramName="paramValue"/> would produce. HTTP basic auth
1083 is used for login/password (user must be one known to the current user manager).
1084 Added: New command 'ca-profiles'. Lists all profiles accessible by the currently authenticated user.
1085 Added: New parameter 'profile'. Allows selecting one of the available profiles rather than getting info for all.
1086 (Mainly for commands cws-connectors, proxy-users and tv-services).
1087 Added: Extra fields in the replies for cws-connectors and proxy-users. Profile is shown where applicable, client-id
1088 is shown for users and metric for connectors. Ecm and emm count added to both connectors and user sessions.
1089 Added: The beginnings of privacy/security. Only users marked as admin will be able to list all users, others will
1090 just get their own information. Note that even admins will be subject to profile restrictions.
1091 Only admins will see the host/ip for the cws connectors, but the rest of the information is available to all.
1092
1093- Changes to the clustered cache implementation: (see example proxy.xml for details)
1094 Added: Example p2p type cluster management. Cache can be configured to get a list of other proxies (host:port) from
1095 a preconfigured "tracker" url. The list is a static plain text file that can be stored anywhere in public.
1096 As a privacy feature, the list can be blowfish encrypted with a configured key that all proxies would need
1097 to have. The file format is one proxy host:port per line, lines starting with # are considered comments.
1098 To encrypt the list file, use: java -jar lib/trtest.jar inputfile.txt outputfile.enc secretkey
1099 The encryption is the same type of blowfish used by fish/mircryption for encrypted irc.
1100 Added: <tracker-url> to section cache-config. URL pointing to a file with a list of peers in host:port format.
1101 Added: <tracker-key> to section cache-config. Blowfish encryption key, if this is set the list file must be
1102 encrypted with the same key.
1103 Added: <tracker-update> to section cache-config. The update interval in minutes, minimum 5.
1104 Added: <local-host> to section cache-config. The cache needs to be able to identify itself in the list of peers,
1105 otherwise it would send updates to itself which would be bad. Set the local-host to the same as this node
1106 has in the list of peers file. If omitted the cache will use the local IP.
1107
1108- UserManager interface now has limits that can be imposed on idividual users, but SimpleUserManager will _not_ use this.
1109 For anyone working on their own user manager, the methods are:
1110 Set getAllowedServices(String name, String profile); // return Set of Integer (service ids, null for all)
1111 Set getAllowedConnectors(String name); // return Set of String (connector names, null for all)
1112 int getAllowedEcmRate(String user); // return minimum interval between ecm in seconds (-1 for no limit)
1113 A note about allowed connectors: this will stop ecm's from this user from being routed to other connectors, but it
1114 will _not_ stop them from watching services that only exist on those excluded connectors, through the cache.
1115
1116- Proxy now detects the type of client for each connected user session.
1117- Fixed xml-related problems with http/xml api and java 1.5.
1118- "Webgui" cleanup, removed everything not used or not working.
1119- Fixes to startup sequence, all errors shown by the start script should now halt the proxy with a non-zero exit code.
1120- Fixed enigma services file parsing (0xffffff transponder ids and other values caused it to fail).
1121
11220.4.9 - First pre-release for testing
1123
1124- Pluggable implementations for user manager, cache handler and filters/loggers. Contact bowman on efnet for details.
1125- Extensible remote api (java rmi and arbitrary http/xml) for monitoring and remote control.
Note: See TracBrowser for help on using the repository browser.