You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

JsonUtils.java 10KB

Ticket tracker with patchset contributions A basic issue tracker styled as a hybrid of GitHub and BitBucket issues. You may attach commits to an existing ticket or you can push a single commit to create a *proposal* ticket. Tickets keep track of patchsets (one or more commits) and allow patchset rewriting (rebase, amend, squash) by detecing the non-fast-forward update and assigning a new patchset number to the new commits. Ticket tracker -------------- The ticket tracker stores tickets as an append-only journal of changes. The journals are deserialized and a ticket is built by applying the journal entries. Tickets are indexed using Apache Lucene and all queries and searches are executed against this Lucene index. There is one trade-off to this persistence design: user attributions are non-relational. What does that mean? Each journal entry stores the username of the author. If the username changes in the user service, the journal entry will not reflect that change because the values are hard-coded. Here are a few reasons/justifications for this design choice: 1. commit identifications (author, committer, tagger) are non-relational 2. maintains the KISS principle 3. your favorite text editor can still be your administration tool Persistence Choices ------------------- **FileTicketService**: stores journals on the filesystem **BranchTicketService**: stores journals on an orphan branch **RedisTicketService**: stores journals in a Redis key-value datastore It should be relatively straight-forward to develop other backends (MongoDB, etc) as long as the journal design is preserved. Pushing Commits --------------- Each push to a ticket is identified as a patchset revision. A patchset revision may add commits to the patchset (fast-forward) OR a patchset revision may rewrite history (rebase, squash, rebase+squash, or amend). Patchset authors should not be afraid to polish, revise, and rewrite their code before merging into the proposed branch. Gitblit will create one ref for each patchset. These refs are updated for fast-forward pushes or created for rewrites. They are formatted as `refs/tickets/{shard}/{id}/{patchset}`. The *shard* is the last two digits of the id. If the id < 10, prefix a 0. The *shard* is always two digits long. The shard's purpose is to ensure Gitblit doesn't exceed any filesystem directory limits for file creation. **Creating a Proposal Ticket** You may create a new change proposal ticket just by pushing a **single commit** to `refs/for/{branch}` where branch is the proposed integration branch OR `refs/for/new` or `refs/for/default` which both will use the default repository branch. git push origin HEAD:refs/for/new **Updating a Patchset** The safe way to update an existing patchset is to push to the patchset ref. git push origin HEAD:refs/heads/ticket/{id} This ensures you do not accidentally create a new patchset in the event that the patchset was updated after you last pulled. The not-so-safe way to update an existing patchset is to push using the magic ref. git push origin HEAD:refs/for/{id} This push ref will update an exisitng patchset OR create a new patchset if the update is non-fast-forward. **Rebasing, Squashing, Amending** Gitblit makes rebasing, squashing, and amending patchsets easy. Normally, pushing a non-fast-forward update would require rewind (RW+) repository permissions. Gitblit provides a magic ref which will allow ticket participants to rewrite a ticket patchset as long as the ticket is open. git push origin HEAD:refs/for/{id} Pushing changes to this ref allows the patchset authors to rebase, squash, or amend the patchset commits without requiring client-side use of the *--force* flag on push AND without requiring RW+ permission to the repository. Since each patchset is tracked with a ref it is easy to recover from accidental non-fast-forward updates. Features -------- - Ticket tracker with status changes and responsible assignments - Patchset revision scoring mechanism - Update/Rewrite patchset handling - Close-on-push detection - Server-side Merge button for simple merges - Comments with Markdown syntax support - Rich mail notifications - Voting - Mentions - Watch lists - Querying - Searches - Partial miletones support - Multiple backend options
10 years ago
Ticket tracker with patchset contributions A basic issue tracker styled as a hybrid of GitHub and BitBucket issues. You may attach commits to an existing ticket or you can push a single commit to create a *proposal* ticket. Tickets keep track of patchsets (one or more commits) and allow patchset rewriting (rebase, amend, squash) by detecing the non-fast-forward update and assigning a new patchset number to the new commits. Ticket tracker -------------- The ticket tracker stores tickets as an append-only journal of changes. The journals are deserialized and a ticket is built by applying the journal entries. Tickets are indexed using Apache Lucene and all queries and searches are executed against this Lucene index. There is one trade-off to this persistence design: user attributions are non-relational. What does that mean? Each journal entry stores the username of the author. If the username changes in the user service, the journal entry will not reflect that change because the values are hard-coded. Here are a few reasons/justifications for this design choice: 1. commit identifications (author, committer, tagger) are non-relational 2. maintains the KISS principle 3. your favorite text editor can still be your administration tool Persistence Choices ------------------- **FileTicketService**: stores journals on the filesystem **BranchTicketService**: stores journals on an orphan branch **RedisTicketService**: stores journals in a Redis key-value datastore It should be relatively straight-forward to develop other backends (MongoDB, etc) as long as the journal design is preserved. Pushing Commits --------------- Each push to a ticket is identified as a patchset revision. A patchset revision may add commits to the patchset (fast-forward) OR a patchset revision may rewrite history (rebase, squash, rebase+squash, or amend). Patchset authors should not be afraid to polish, revise, and rewrite their code before merging into the proposed branch. Gitblit will create one ref for each patchset. These refs are updated for fast-forward pushes or created for rewrites. They are formatted as `refs/tickets/{shard}/{id}/{patchset}`. The *shard* is the last two digits of the id. If the id < 10, prefix a 0. The *shard* is always two digits long. The shard's purpose is to ensure Gitblit doesn't exceed any filesystem directory limits for file creation. **Creating a Proposal Ticket** You may create a new change proposal ticket just by pushing a **single commit** to `refs/for/{branch}` where branch is the proposed integration branch OR `refs/for/new` or `refs/for/default` which both will use the default repository branch. git push origin HEAD:refs/for/new **Updating a Patchset** The safe way to update an existing patchset is to push to the patchset ref. git push origin HEAD:refs/heads/ticket/{id} This ensures you do not accidentally create a new patchset in the event that the patchset was updated after you last pulled. The not-so-safe way to update an existing patchset is to push using the magic ref. git push origin HEAD:refs/for/{id} This push ref will update an exisitng patchset OR create a new patchset if the update is non-fast-forward. **Rebasing, Squashing, Amending** Gitblit makes rebasing, squashing, and amending patchsets easy. Normally, pushing a non-fast-forward update would require rewind (RW+) repository permissions. Gitblit provides a magic ref which will allow ticket participants to rewrite a ticket patchset as long as the ticket is open. git push origin HEAD:refs/for/{id} Pushing changes to this ref allows the patchset authors to rebase, squash, or amend the patchset commits without requiring client-side use of the *--force* flag on push AND without requiring RW+ permission to the repository. Since each patchset is tracked with a ref it is easy to recover from accidental non-fast-forward updates. Features -------- - Ticket tracker with status changes and responsible assignments - Patchset revision scoring mechanism - Update/Rewrite patchset handling - Close-on-push detection - Server-side Merge button for simple merges - Comments with Markdown syntax support - Rich mail notifications - Voting - Mentions - Watch lists - Querying - Searches - Partial miletones support - Multiple backend options
10 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347
  1. /*
  2. * Copyright 2011 gitblit.com.
  3. *
  4. * Licensed under the Apache License, Version 2.0 (the "License");
  5. * you may not use this file except in compliance with the License.
  6. * You may obtain a copy of the License at
  7. *
  8. * http://www.apache.org/licenses/LICENSE-2.0
  9. *
  10. * Unless required by applicable law or agreed to in writing, software
  11. * distributed under the License is distributed on an "AS IS" BASIS,
  12. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  13. * See the License for the specific language governing permissions and
  14. * limitations under the License.
  15. */
  16. package com.gitblit.utils;
  17. import java.io.BufferedReader;
  18. import java.io.IOException;
  19. import java.io.InputStream;
  20. import java.io.InputStreamReader;
  21. import java.io.OutputStream;
  22. import java.lang.reflect.Type;
  23. import java.net.HttpURLConnection;
  24. import java.net.URLConnection;
  25. import java.text.DateFormat;
  26. import java.text.ParseException;
  27. import java.text.SimpleDateFormat;
  28. import java.util.Collection;
  29. import java.util.Date;
  30. import java.util.Locale;
  31. import java.util.Map;
  32. import java.util.TimeZone;
  33. import com.gitblit.Constants.AccessPermission;
  34. import com.gitblit.GitBlitException.ForbiddenException;
  35. import com.gitblit.GitBlitException.NotAllowedException;
  36. import com.gitblit.GitBlitException.UnauthorizedException;
  37. import com.gitblit.GitBlitException.UnknownRequestException;
  38. import com.gitblit.models.RepositoryModel;
  39. import com.gitblit.models.UserModel;
  40. import com.google.gson.ExclusionStrategy;
  41. import com.google.gson.FieldAttributes;
  42. import com.google.gson.Gson;
  43. import com.google.gson.GsonBuilder;
  44. import com.google.gson.JsonDeserializationContext;
  45. import com.google.gson.JsonDeserializer;
  46. import com.google.gson.JsonElement;
  47. import com.google.gson.JsonPrimitive;
  48. import com.google.gson.JsonSerializationContext;
  49. import com.google.gson.JsonSerializer;
  50. import com.google.gson.JsonSyntaxException;
  51. import com.google.gson.reflect.TypeToken;
  52. /**
  53. * Utility methods for json calls to a Gitblit server.
  54. *
  55. * @author James Moger
  56. *
  57. */
  58. public class JsonUtils {
  59. public static final Type REPOSITORIES_TYPE = new TypeToken<Map<String, RepositoryModel>>() {
  60. }.getType();
  61. public static final Type USERS_TYPE = new TypeToken<Collection<UserModel>>() {
  62. }.getType();
  63. /**
  64. * Creates JSON from the specified object.
  65. *
  66. * @param o
  67. * @return json
  68. */
  69. public static String toJsonString(Object o) {
  70. String json = gson().toJson(o);
  71. return json;
  72. }
  73. /**
  74. * Convert a json string to an object of the specified type.
  75. *
  76. * @param json
  77. * @param clazz
  78. * @return an object
  79. */
  80. public static <X> X fromJsonString(String json, Class<X> clazz) {
  81. return gson().fromJson(json, clazz);
  82. }
  83. /**
  84. * Convert a json string to an object of the specified type.
  85. *
  86. * @param json
  87. * @param clazz
  88. * @return an object
  89. */
  90. public static <X> X fromJsonString(String json, Type type) {
  91. return gson().fromJson(json, type);
  92. }
  93. /**
  94. * Reads a gson object from the specified url.
  95. *
  96. * @param url
  97. * @param type
  98. * @return the deserialized object
  99. * @throws {@link IOException}
  100. */
  101. public static <X> X retrieveJson(String url, Type type) throws IOException,
  102. UnauthorizedException {
  103. return retrieveJson(url, type, null, null);
  104. }
  105. /**
  106. * Reads a gson object from the specified url.
  107. *
  108. * @param url
  109. * @param type
  110. * @return the deserialized object
  111. * @throws {@link IOException}
  112. */
  113. public static <X> X retrieveJson(String url, Class<? extends X> clazz) throws IOException,
  114. UnauthorizedException {
  115. return retrieveJson(url, clazz, null, null);
  116. }
  117. /**
  118. * Reads a gson object from the specified url.
  119. *
  120. * @param url
  121. * @param type
  122. * @param username
  123. * @param password
  124. * @return the deserialized object
  125. * @throws {@link IOException}
  126. */
  127. public static <X> X retrieveJson(String url, Type type, String username, char[] password)
  128. throws IOException {
  129. String json = retrieveJsonString(url, username, password);
  130. if (StringUtils.isEmpty(json)) {
  131. return null;
  132. }
  133. return gson().fromJson(json, type);
  134. }
  135. /**
  136. * Reads a gson object from the specified url.
  137. *
  138. * @param url
  139. * @param clazz
  140. * @param username
  141. * @param password
  142. * @return the deserialized object
  143. * @throws {@link IOException}
  144. */
  145. public static <X> X retrieveJson(String url, Class<X> clazz, String username, char[] password)
  146. throws IOException {
  147. String json = retrieveJsonString(url, username, password);
  148. if (StringUtils.isEmpty(json)) {
  149. return null;
  150. }
  151. return gson().fromJson(json, clazz);
  152. }
  153. /**
  154. * Retrieves a JSON message.
  155. *
  156. * @param url
  157. * @return the JSON message as a string
  158. * @throws {@link IOException}
  159. */
  160. public static String retrieveJsonString(String url, String username, char[] password)
  161. throws IOException {
  162. try {
  163. URLConnection conn = ConnectionUtils.openReadConnection(url, username, password);
  164. InputStream is = conn.getInputStream();
  165. BufferedReader reader = new BufferedReader(new InputStreamReader(is,
  166. ConnectionUtils.CHARSET));
  167. StringBuilder json = new StringBuilder();
  168. char[] buffer = new char[4096];
  169. int len = 0;
  170. while ((len = reader.read(buffer)) > -1) {
  171. json.append(buffer, 0, len);
  172. }
  173. is.close();
  174. return json.toString();
  175. } catch (IOException e) {
  176. if (e.getMessage().indexOf("401") > -1) {
  177. // unauthorized
  178. throw new UnauthorizedException(url);
  179. } else if (e.getMessage().indexOf("403") > -1) {
  180. // requested url is forbidden by the requesting user
  181. throw new ForbiddenException(url);
  182. } else if (e.getMessage().indexOf("405") > -1) {
  183. // requested url is not allowed by the server
  184. throw new NotAllowedException(url);
  185. } else if (e.getMessage().indexOf("501") > -1) {
  186. // requested url is not recognized by the server
  187. throw new UnknownRequestException(url);
  188. }
  189. throw e;
  190. }
  191. }
  192. /**
  193. * Sends a JSON message.
  194. *
  195. * @param url
  196. * the url to write to
  197. * @param json
  198. * the json message to send
  199. * @return the http request result code
  200. * @throws {@link IOException}
  201. */
  202. public static int sendJsonString(String url, String json) throws IOException {
  203. return sendJsonString(url, json, null, null);
  204. }
  205. /**
  206. * Sends a JSON message.
  207. *
  208. * @param url
  209. * the url to write to
  210. * @param json
  211. * the json message to send
  212. * @param username
  213. * @param password
  214. * @return the http request result code
  215. * @throws {@link IOException}
  216. */
  217. public static int sendJsonString(String url, String json, String username, char[] password)
  218. throws IOException {
  219. try {
  220. byte[] jsonBytes = json.getBytes(ConnectionUtils.CHARSET);
  221. URLConnection conn = ConnectionUtils.openConnection(url, username, password);
  222. conn.setRequestProperty("Content-Type", "text/plain;charset=" + ConnectionUtils.CHARSET);
  223. conn.setRequestProperty("Content-Length", "" + jsonBytes.length);
  224. // write json body
  225. OutputStream os = conn.getOutputStream();
  226. os.write(jsonBytes);
  227. os.close();
  228. int status = ((HttpURLConnection) conn).getResponseCode();
  229. return status;
  230. } catch (IOException e) {
  231. if (e.getMessage().indexOf("401") > -1) {
  232. // unauthorized
  233. throw new UnauthorizedException(url);
  234. } else if (e.getMessage().indexOf("403") > -1) {
  235. // requested url is forbidden by the requesting user
  236. throw new ForbiddenException(url);
  237. } else if (e.getMessage().indexOf("405") > -1) {
  238. // requested url is not allowed by the server
  239. throw new NotAllowedException(url);
  240. } else if (e.getMessage().indexOf("501") > -1) {
  241. // requested url is not recognized by the server
  242. throw new UnknownRequestException(url);
  243. }
  244. throw e;
  245. }
  246. }
  247. // build custom gson instance with GMT date serializer/deserializer
  248. // http://code.google.com/p/google-gson/issues/detail?id=281
  249. public static Gson gson(ExclusionStrategy... strategies) {
  250. GsonBuilder builder = new GsonBuilder();
  251. builder.registerTypeAdapter(Date.class, new GmtDateTypeAdapter());
  252. builder.registerTypeAdapter(AccessPermission.class, new AccessPermissionTypeAdapter());
  253. if (!ArrayUtils.isEmpty(strategies)) {
  254. builder.setExclusionStrategies(strategies);
  255. }
  256. return builder.create();
  257. }
  258. public static class GmtDateTypeAdapter implements JsonSerializer<Date>, JsonDeserializer<Date> {
  259. private final DateFormat dateFormat;
  260. public GmtDateTypeAdapter() {
  261. dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'", Locale.US);
  262. dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
  263. }
  264. @Override
  265. public synchronized JsonElement serialize(Date date, Type type,
  266. JsonSerializationContext jsonSerializationContext) {
  267. synchronized (dateFormat) {
  268. String dateFormatAsString = dateFormat.format(date);
  269. return new JsonPrimitive(dateFormatAsString);
  270. }
  271. }
  272. @Override
  273. public synchronized Date deserialize(JsonElement jsonElement, Type type,
  274. JsonDeserializationContext jsonDeserializationContext) {
  275. try {
  276. synchronized (dateFormat) {
  277. Date date = dateFormat.parse(jsonElement.getAsString());
  278. return new Date((date.getTime() / 1000) * 1000);
  279. }
  280. } catch (ParseException e) {
  281. throw new JsonSyntaxException(jsonElement.getAsString(), e);
  282. }
  283. }
  284. }
  285. private static class AccessPermissionTypeAdapter implements JsonSerializer<AccessPermission>, JsonDeserializer<AccessPermission> {
  286. private AccessPermissionTypeAdapter() {
  287. }
  288. @Override
  289. public synchronized JsonElement serialize(AccessPermission permission, Type type,
  290. JsonSerializationContext jsonSerializationContext) {
  291. return new JsonPrimitive(permission.code);
  292. }
  293. @Override
  294. public synchronized AccessPermission deserialize(JsonElement jsonElement, Type type,
  295. JsonDeserializationContext jsonDeserializationContext) {
  296. return AccessPermission.fromCode(jsonElement.getAsString());
  297. }
  298. }
  299. public static class ExcludeField implements ExclusionStrategy {
  300. private Class<?> c;
  301. private String fieldName;
  302. public ExcludeField(String fqfn) throws SecurityException, NoSuchFieldException,
  303. ClassNotFoundException {
  304. this.c = Class.forName(fqfn.substring(0, fqfn.lastIndexOf(".")));
  305. this.fieldName = fqfn.substring(fqfn.lastIndexOf(".") + 1);
  306. }
  307. @Override
  308. public boolean shouldSkipClass(Class<?> arg0) {
  309. return false;
  310. }
  311. @Override
  312. public boolean shouldSkipField(FieldAttributes f) {
  313. return (f.getDeclaringClass() == c && f.getName().equals(fieldName));
  314. }
  315. }
  316. }